超越单一语言深度研究：使用跨语言检索功能来评估代理程序和检索器
Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus
摘要
那些能够进行深度研究的智能体，其评估标准主要体现在它们寻找证据、对所检索到的信息进行分析，以及生成有根据的回答的能力上。不过，现有的搜索基准测试大多假设用户的查询内容以及相关的证据都是用同一种语言编写的，因此无法判断当相关证据以另一种语言呈现时，智能体搜索系统是否能够正常运作。我们提出了XBCP（Cross-lingual BrowseComp-Plus）这一新的测试框架——该框架保留了BrowseComp-Plus中英文问答模式，但改变了相关文档的语言形式。XBCP模拟了两种不同的测试环境：在跨语言模式下，每个查询都对应着一种指定语言的证据；而在多语言模式下，所有证据被随机分配到12种不同语言中，这些语言分别对应于资源丰富或匮乏的不同情境。我们通过使用稀疏式和密集式多语言检索工具来评估四种深度研究智能体，并测量了它们的答案准确性、证据回忆能力、搜索行为、校准效果、引用可靠性以及“神谕式”检索能力。结果表明，当证据被翻译成其他语言时，智能体的性能会显著下降。即便是那些性能强大的智能体，也仍然难以准确回忆起相关证据，且其校准效果也会变差，引用的证据也变得不够可靠。值得注意的是，即使所有关键证据都被直接提供出来，智能体的准确性依然较低。这些发现表明，跨语言深度研究既存在检索失败的问题，同时也存在智能体在处理不匹配语言的证据时的困难。
English Abstract
Deep research agents are increasingly evaluated on their ability to search for evidence, reason over retrieved sources, and produce grounded answers. Existing browsing benchmarks, however, largely assume that the user's query and the supporting evidence are written in the same language, leaving open whether agentic search systems can operate when relevant evidence appears in another language. We introduce XBCP (Cross-lingual BrowseComp-Plus), a controlled benchmark that preserves the English question-and-answer space of BrowseComp-Plus but varies the languages of the supporting documents. XBCP instantiates two complementary settings: in the cross-lingual setting, each query is paired with evidence in a single assigned language. In the multilingual setting, the full evidence corpus is distributed equally and randomly across 12 languages spanning high-resource and low-resource regimes. We evaluate four deep research agents using sparse and dense multilingual retrievers, measuring answer accuracy, evidence recall, search behavior, calibration, citation fidelity, and oracle retrieval. Results reveal substantial degradation when evidence is translated. Even strong, dense retrievers lose evidence recall, and agents become less calibrated and cite evidence less reliably. Notably, accuracy remains lower even when all gold evidence is supplied directly. These findings suggest that cross-lingual deep research exposes both retrieval failures and an independent, agent-side difficulty in integrating language-mismatched evidence.