用于可靠视频理解的信任感知工具调度机制
Confidence-Aware Tool Orchestration for Robust Video Understanding
摘要
视频推理语言模型默认认为每个输入帧都具有相同的可靠性。这就导致了我们所说的“盲目信任问题”：在诸如运动模糊、眩光或遮挡等现实中的干扰情况下，先进的视频推理模型在实际测试中的准确率可能会下降15-30%，而它们却无法意识到自己的视觉证据已经受到破坏。为了应对这一挑战，我们提出了Robust-TO这一智能视频理解框架，该框架将每帧的可靠性因素明确纳入到推理的各个阶段中。Robust-TO将不同的视觉感知工具整合在一个统一的证据接口下。每种工具都会收到来自原始问题的子查询，以及根据可靠性相关得分选出的可靠帧集。它以统一的形式提供证据：具体的预测结果（如边界框、运动轨迹、识别出的文本或动作标签）、时间信息以及经过校准的可靠性分数。在推理过程中，这些经过校准的分数用于指导证据权重分配，从而实现三层合成过程（高/中/低），同时还能生成一种兼顾正确性、证据可靠性和效率的置信度-成本GRPO奖励机制。在涵盖八项任务的两个视频推理测试集上，Robust-TO在干净输入数据上的平均准确率达到了56.4%，比最强大的开源基线高出10.6%，也优于Gemini-2.5-Pro的46.2%。在五种现实中的损坏类型下，Robust-TO仍然保持54.3%的平均准确率，比最强大的开源基线高出5.8%，而且在所有对比方法中，其从干净状态到损坏状态的准确率下降幅度最小。
English Abstract
Video reasoning language models implicitly assume that every input frame is equally reliable. This leads to what we term the Blind Trust Problem: under realistic perturbations such as motion blur, glare, or occlusion, frontier video reasoning models can suffer 15-30%p accuracy drops on real-world embodied benchmarks, while remaining unaware that their visual evidence has been degraded. To address this challenge, we propose Robust-TO, an agentic video understanding framework that explicitly integrates per-frame trustworthiness into every stage of reasoning. Robust-TO organizes heterogeneous visual perception tools under a unified evidence interface. Each tool receives a sub-query derived from the original question and a set of trustworthy frames selected by the reliability-relevance score. It returns evidence in a shared format: a concrete prediction (e.g., a bounding box, motion trajectory, recognized text, or action label), temporal grounding, and a calibrated reliability score. During reasoning, these calibrated scores guide evidence weighting in a three-tier synthesis process (high/medium/low) and define a confidence-cost GRPO reward that jointly optimizes correctness, evidence reliability, and efficiency. On two video reasoning benchmarks spanning eight tasks, Robust-TO achieves 56.4% average accuracy on clean inputs, surpassing the strongest open-source baseline by 10.6%p and outperforming Gemini-2.5-Pro (46.2%). Under five realistic corruption types, Robust-TO maintains 54.3% average accuracy, 5.8%p above the strongest open-source baseline, while exhibiting the smallest clean-to-corrupted accuracy drop among all compared methods.