在空间视觉语言模型中强化双路径推理
Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
摘要
空间型虚拟语言模型在几何感知方面取得了显著进展,但需要结合深度、距离和场景关系进行多步骤推理的复杂空间推理仍然具有挑战性。此外,不同的空间查询需要不同的策略:有些问题可以通过纯语言层面的逐步推导来解决,而另一些则需要在进行定量推理之前先明确3D信息。我们提出了“基于强化学习的双路径空间推理方法”——该框架为空间型虚拟语言模型提供了两种互补的推理方式:仅依赖语言的推理方式,以及“检测后推理”方式。前者通过逐步的语言推理来解决问题,后者则通过区域标记来识别3D几何特征,然后再进行几何推理。SR-REAL首先通过监督式微调过程来构建这两种推理方式,同时提供区域与3D信息的连接机制;随后通过强化学习来优化策略模型,以准确度和格式作为奖励指标。对于“检测后推理”方式,基于离散中心的检测奖励可以进一步优化几何对齐效果。在多种空间基准测试中,SR-REAL的表现明显优于其他空间型虚拟语言模型的基线模型:(i)一个单一的强化学习训练模型可以同时支持两种推理方式,其中“检测后推理”方式在需要区域感知的任务中表现更好,而“仅依赖语言推理”方式则能提升一般的空间推理能力;(ii)同时训练这两种方式可以促进相互间的优化;(iii)高质量、混合化的冷启动数据对于稳定的强化学习优化至关重要;(iv)该模型能够在不同数据集和领域中保持性能稳定,无需针对每个任务进行调整,从而证明了“仅依赖语言推理”和“检测后推理”两种方式之间的有效迁移。
English Abstract
Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging. Moreover, different spatial queries call for fundamentally different strategies: some are best addressed through purely linguistic, step-by-step deduction, while others require explicit 3D grounding before quantitative inference. We present Dual-Path Spatial Reasoning via Reinforcement Learning for Spatial VLMs (SR-REAL), a unified framework that equips a spatial VLM with two complementary reasoning paths: Language-Only Reasoning (LOR), which performs step-by-step linguistic deduction, and Detect-Then-Reason (DTR), which detects 3D geometric cues (e.g., centers or bounding boxes) via region tokens before explicit geometric inference. SR-REAL begins with a cold-start supervised fine-tuning stage that constructs LOR and DTR chain-of-thought supervision and exposes a region-to-3D interface, followed by RL that optimizes the policy model with accuracy and format rewards; for DTR, a discrete center-based detection reward further refines geometric alignment. Across diverse spatial benchmarks, SR-REAL significantly outperforms spatial VLM baselines: (i) a single RL-trained model supports both reasoning paths, with DTR excelling in region-aware tasks through precise 3D localization and LOR enhancing general spatial reasoning; (ii) jointly training both paths fosters mutual reinforcement; (iii) high-quality, blended cold-start data is crucial for stable RL optimization; and (iv) the model generalizes across datasets and domains without per-task tuning, demonstrating positive transfer between LOR and DTR.