在推理之前进行感知：分离感知与推理，以实现对捷径具有抵抗力的多模态在线策略自蒸馏
Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation
摘要
基于策略的自复制训练方法（OPSD）通过让模型使用自己的生成结果来训练模型，并使用冻结的版本作为参考目标，从而提供丰富的令牌级目标。这种方法非常适合用于LLM的推理过程，但将其直接应用于多模态大语言模型时可能会产生问题：因为特权目标可能主要依据文本参考目标而非图像来指导令牌的生成。因此，我们提出了ViGOS这一针对多模态大语言模型训练后使用的视觉导向OPSD框架。该框架中，模型首先生成视觉描述，然后基于该描述进行推理以得到最终答案。对于有效的生成结果，仅使用图像感知模块来监督描述生成过程，而特权推理模块则负责监督推理和最终答案的生成。对于无效的生成结果，则使用参考模块来恢复正确的输出格式。在各种通用视觉-语言任务、专家推理任务、视觉数学任务、空间定位任务以及视觉-语言先验评估中，ViGOS能够保留OPSD的主要优势，同时改善在容易采用捷径的场景下模型对图像的响应能力。
English Abstract
On-policy self-distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense token-level targets conditioned on a reference target. This works well for LLM reasoning, but a direct extension to multimodal large language models (MLLMs) can create a shortcut: the privileged target may guide tokens mainly based on the text reference target rather than the image. We propose ViGOS, a visually grounded OPSD framework for MLLM post-training. The student first writes a visual description and then reasons toward the final answer. For valid rollouts, an image-only perception teacher supervises the description, while a privileged reasoning teacher supervises the reasoning and final answer on the same student prefix. A reference teacher is used only for invalid rollouts to recover the output format. Across general vision-language, expert reasoning, visual math, spatial grounding, and visual-language-prior benchmarks, ViGOS keeps the main benefits of OPSD and improves image-grounded behavior in shortcut-prone settings.