语言模型结合何时能发挥作用？在67种前沿模型中，路由、投票和混合智能体任务上的共同失败上限问题
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
摘要
诸如路由、投票、级联、融合以及混合代理等多模型LLM系统被用来超越单一模型的性能水平。我们发现，这些系统的性能提升受到一个很少被提及的参数的限制。对于那些输出为单个模型答案的政策来说，准确率不可能超过1减去β值，其中β表示每个模型在相同查询下出错的频率。相比之下，常用的诊断指标——平均成对误差相关性ρ——无法揭示β值：即使各模型的边际分布和成对相关性相同，它们的整体出错率也可能不同。对β值的Clopper-Pearson界限分析表明，任何路由器、投票机制或级联系统在训练之前所能获得的性能提升都是有限的。 在21家供应商提供的67种模型中，经过四元正态校准的单因子模型仍然低估了整体出错情况：在开放数学中，观察到的β值为0.052，而完整的67种模型的高斯copula模型下的β值为0.023，相当于低了约2.5倍。90%的置信区间范围为1.7到3.4，k值为17。这种效应同样适用于执行分级代码，此时β值为0.079。如果以自由回答的形式重新提出相同的GPQA-Diamond问题，而不是选择题形式，那么β值变为0.127，而由五名法官组成的评审小组则得到κ值为0.73到0.92的结果，这表明错误发生在答案本身上，而非主题本身。在质量相当的情况下，低ρ值的异构模型组合优于高ρ值的Self-MoA模型；但在我们样本中的可检查任务中，组合多个模型往往无法超越最好的单个模型，除非有强烈的查询级路由信号。性能提升来自模型在不同问题上的失败，而非增加更多模型。
English Abstract
Multi-model LLM systems such as routing, voting, cascades, fusion, and mixture-of-agents are used to beat single-model accuracy. We show that their gain is capped by a quantity the field rarely reports. For any policy whose output is one member model answer, accuracy cannot exceed one minus beta, where beta is the rate at which every model is wrong on the same query. In contrast, the usual diagnostic, average pairwise error correlation rho, cannot identify beta: error laws with identical marginals and pairwise correlations can have different all-wrong rates. A Clopper-Pearson bound on beta gives a finite-sample certificate on the largest gain any router, vote, or cascade could deliver before training a router. Across 67 models from 21 providers, a tetrachoric-calibrated single-factor model still underprices the all-wrong tail: on open-ended mathematics, observed beta is 0.052 versus 0.023 under the full 67-model Gaussian copula, about 2.5 times underpricing, with 90 percent CI 1.7 to 3.4 and k equals 17. The effect recurs on execution-graded code, where beta is 0.079. Re-asking the same GPQA-Diamond questions in free-response rather than multiple-choice form reopens the tail, with beta 0.127 and a five-judge panel with kappa 0.73 to 0.92, locating co-failure in answer format rather than subject. At matched quality, low-rho heterogeneous ensembles beat high-rho Self-MoA, but on checkable tasks in our pool, combining models rarely beats the single best model without a strong query-level routing signal. Gains come from models failing on different questions, not from adding more models.