ChartWalker：对跨图表RAG任务进行基准测试
ChartWalker: Benchmarking the Cross-Chart RAG Task
摘要
跨图表检索增强生成技术对于科学、商业和政治领域中的复杂多模态分析任务至关重要。然而，现有的评估基准要么专注于结构清晰且以文本形式呈现的表格数据，要么只是通过提取关键信息来生成跨图表问题，这种方式往往会导致查询与证据之间的词汇重叠，进而产生逻辑上不一致的推理过程。为了解决这一问题，我们提出了ChartWalker这一新型框架，用于构建具有挑战性的跨图表RAG任务。ChartWalker采用一种针对图表设计的分层知识图谱构建方法，能够按照不同的粒度对实体和关系进行组织，从而保持分析结构的完整性。此外，我们还提出了一种注重结构意识的采样算法，该算法能够生成语义上连贯的多层级推理路径，从而可以明确控制查询的复杂度和粒度，进而实现有效的问答生成。基于这一框架，我们发布了ChartWalker-Bench这一全面的评估基准，涵盖了多种领域和不同类型的跨图表查询。对各种RAG方法的广泛测试表明，各方法的性能存在明显差异，这凸显了该评估基准的实用性和重要性。此外，我们还开发了ChartWalker-Agent这一智能代理系统，有助于促进分析工作，并为未来的系统设计提供灵感。
English Abstract
Cross-Chart Retrieval-Augmented Generation (RAG) is critical for complex multi-modal analytical tasks in scientific, business, and political domains. However, existing benchmarks either focus on tables, which are well-structured and textualized, or generate cross-chart questions by simply extracting key points, which often induces lexical overlap between queries and evidence and yields logically inconsistent reasoning chains. To address this, we introduce ChartWalker, a novel framework for constructing challenging cross-chart RAG tasks. ChartWalker features a hierarchical knowledge graph construction method tailored to charts, which organizes entities and relations by granularity to preserve analytical structure. We then propose a structure-aware sampling algorithm that synthesizes semantically coherent, multi-hop reasoning paths, enabling explicit control over query difficulty and granularity for QA generation. Built with this framework, we release ChartWalker-Bench, a comprehensive benchmark spanning diverse domains and cross-chart query types. Extensive evaluations across major RAG paradigms reveal significant performance gaps, underscoring the benchmark's difficulty and utility. Furthermore, we provide ChartWalker-Agent, an agentic baseline to facilitate analysis and inspire future system design.