V-Zero：基于对比性证据门控的无需答案标签的在线策略蒸馏方法，用于精细视觉推理
V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning
摘要
精细化的视觉推理需要多模态大语言模型来识别与任务相关的视觉证据，并将推理基于局部图像区域进行实现。现有的智能体方法通常依赖带有可验证奖励的强化学习，或者基于大规模标注的推理数据进行监督式微调，这会导致昂贵的探索过程、手工设计的验证规则，或者过度依赖文本监督。避免这种外部答案标签的一种自然方式就是从学生自己采样的轨迹中学习，这就指向了“策略内蒸馏”方法。为了了解“策略内蒸馏”在视觉推理中能做什么、不能做什么，我们将它视为一种无负样本的停止梯度对齐方法。这种视角表明，虽然“策略内蒸馏”能够提供有效的令牌级修正，但其效果受到轨迹级区分能力不足的限制。基于这些观察结果，我们提出了V-Zero——一种无需答案标签的视觉推理框架，该框架利用对比证据机制来实现蒸馏。V-Zero不使用任何标注的文本答案标签；在训练过程中，它将与问题相关的区域与负面视觉场景配对，以评估学生采样的轨迹，从而实现密集的令牌级蒸馏。在多个视觉推理基准测试中的实验表明，V-Zero能够持续改善精细化的视觉推理能力，同时保持良好的泛化能力。值得注意的是，V-Zero的速度比之前的监督式微调方法快5倍以上，比强化学习方法的基线方法快10倍以上。代码和数据集将在https://github.com/eVI-group-SCU/V-Zero上发布。
English Abstract
Fine-grained visual reasoning requires multimodal large language models (MLLMs) to identify task-relevant visual evidence and ground their reasoning in local image regions. Existing agentic methods typically rely on reinforcement learning with verifiable rewards or supervised fine-tuning on large-scale annotated reasoning traces, leading to costly exploration, hand-designed verification rules, or heavy dependence on textual supervision. A natural way to avoid such external answer labels is to learn from trajectories sampled by the student itself, which points to On-Policy Distillation (OPD). To understand what OPD can and cannot provide for visual reasoning, we revisit it as negative-free stop-gradient alignment. This perspective shows that, although OPD provides effective token-level correction, its ceiling is constrained by the absence of trajectory-level discrimination. Motivated by these observations, we propose V-Zero, an answer-label-free framework for visual reasoning with contrastive evidence gating. V-Zero uses no annotated textual answer labels; instead, during training it pairs a question-relevant regional crop with a negative visual view to evaluate student-sampled trajectories and gate dense token-level distillation. Experiments on multiple visual reasoning benchmarks show that V-Zero consistently improves fine-grained visual reasoning while preserving strong generalization. Notably, V-Zero is more than 5times faster than previous supervised fine-tuning methods and more than 10times faster than reinforcement learning baselines. Code and dataset will be released at https://github.com/eVI-group-SCU/V-Zero