IndustryBench-MIPU：工业产品多图像属性值提取的基准测试
IndustryBench-MIPU: Benchmarking Multi-Image Attribute Value Extraction for Industrial Products
摘要
诸如阀门和断路器这样的工业产品，其技术规格非常复杂，这些规格决定了采购、兼容性和安全性等问题在供应链中的处理方式。这些规格分散在多种不同的产品信息中，包括规格表、标签以及技术图纸等。不过，多模态大语言模型是否能够可靠地提取出这些规格，目前仍缺乏研究。为了填补这一空白，我们提出了IndustryBench-MIPU——这是首个用于多图像工业产品理解的大规模基准测试工具。该工具基于结构化属性提取技术，能够从产品图像中识别出各种属性与对应值的对应关系。这项任务涉及对规格表和标签上的文本进行识别、对技术图纸进行视觉推理、利用领域知识来理解工业术语，以及整合不同图像中的信息以完整获取所有相关规格。具体来说，该基准测试包含4,559种产品，分布在27,652张图像上，共有103,703条注释，这些注释涉及18个工业领域。该基准测试是通过多模型共识和三层质量保证机制构建而成的。在单图像和多图像环境下对9种大语言模型进行评估后发现，这些模型的完整性存在明显差距：虽然这些模型具有较高的准确率（86–94%），但其中最好的模型只能恢复49.9%的产品级属性；从单图像提取到多图像提取，召回率会下降15–34个百分点。核心瓶颈在于多图像的完整性，而非单图像的准确性。该数据集和代码均可公开获取。
English Abstract
Industrial products such as valves and circuit breakers are defined by dense technical specifications that govern procurement, compatibility, and safety across supply chains. These specifications are scattered across multiple heterogeneous product images, including specification tables, nameplates, and technical drawings, yet whether Multimodal Large Language Models (MLLMs) can reliably recover them remains underexplored. To fill this gap, we introduce IndustryBench-MIPU, the first large-scale benchmark for multi-image industrial product understanding, built around structured attribute extraction -- recovering property-value pairs from product images. This task jointly probes text recognition on specification tables and nameplates, visual reasoning over technical drawings, domain knowledge to decode industrial terminology, and cross-image evidence integration to assemble scattered specifications. Concretely, the benchmark comprises 4,559 products across 27,652 images with 103,703 annotations spanning 18 industrial categories, constructed through multi-model consensus and three-tier quality assurance. Evaluating nine MLLMs under both single-image and product-level multi-image settings reveals a stark completeness gap: models achieve high precision (86--94%) but the best recovers only 49.9% of product-level attributes; moving from single-image to multi-image extraction costs 15--34 percentage points of recall. Multi-image completeness, not single-image accuracy, is the core bottleneck. Dataset and code are publicly available.