Ko-WideSearch：一种用于网络代理进行全面集合枚举的韩国式广度搜索基准测试
Ko-WideSearch: A Korean Breadth-Search Benchmark for Exhaustive Set Enumeration by Web Agents
摘要
Web-agent测试工具大多用于衡量“深度”——即把某个复杂的答案隐藏在一系列约束条件之中；而“广度”则很少被评估，因为需要全面列出所有元素并记录每个元素的属性，尤其是在非英语语言环境中。衡量广度也相当困难：确保一个“黄金集合”是完整的、且每个元素都正确，其成本远高于只验证一个答案的成本。我介绍了Ko-WideSearch这一韩国式的广度搜索测试工具，该工具由自动化的合成与验证流程构建而成。每个任务都会指定一个集合中的特定实体——比如电视季、朝代、联赛、行政区域或选举——然后要求提供该实体的完整成员列表以及每个元素的属性表，这些属性根据Item、Column和Row-F1进行评分。该测试工具涵盖了190个实体和16个类别，涉及228张表格，分为三个难度级别。这两个参数可以通过我独立调整的两种方式来设定——表格宽度和二维复合键——因此，不同级别的跨乘积关系可以从0%提升到100%。在构建和评分过程中，使用同一个考虑归一化的比较器，因此日期和数量等稳定列不会因格式问题而被过度处理。在20个Web-agent中，失败情况都是一致的：代理能够恢复整个集合，但无法恢复具体的行数据（例如，Item-F1为92.8，而Row-F1仅为53.7）。随着参数的变化，准确率也会持续下降，而无论是增加搜索次数还是处理时间，都无法缩小差距。从单个单元格来看，难点在于找到正确的数值，而不是对其进行格式化：那些没有固定格式的自由文本单元格往往会出现问题，而那些有标准答案的单元格，比如日期或名称，通常就能得到正确的结果。
English Abstract
Web-agent benchmarks overwhelmingly measure depth -- pinning one obscure answer behind a chain of constraints -- while breadth, exhaustively enumerating a closed set and filling each item's attributes, is barely evaluated, especially outside English. Breadth is also hard to build: certifying that a gold set is complete and every cell correct is far costlier than checking a single answer. I introduce Ko-WideSearch, a Korean breadth-search benchmark built by an automated synthesize-and-verify pipeline. Each task names a set-parent entity -- a TV season, a dynasty, a league, an administrative region, an election -- and asks for its full membership plus a per-item attribute table, graded by Item-, Column-, and Row-F1. It spans 228 tables over 190 entities and sixteen categories across three difficulty tiers, set by two structural knobs I dial independently -- table width and a 2-D composite key -- so cross-product membership climbs from 0\% to 100\% across the tiers. A single normalization-aware comparator is shared between gold construction and grading, so stable date and count columns are not over-dropped on formatting alone. Across twenty web agents, the failure is consistent: agents recover the set but not the rows (e.g.\ Item-F1 92.8 against Row-F1 53.7), accuracy falls steadily as the knobs harden, and neither more search nor more spend closes the gap. Broken down by cell, the hard part is finding the right value, not formatting it: open-ended free-text cells fail most, while cells with a standard answer such as a date or a name usually come out right.