GeneralVLA-2：基于几何信息的重建与受控记忆技术，用于机器人规划
GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning
摘要
通用型视觉-语言-动作系统需要以物体为中心的3D信息以及可重复使用的操作经验，才能规划出可靠的机器人运动路径。GeneralVLA提供了一个分层式接口，用于将语言和RGB-D数据转换为3D操作路径，不过仍然存在两个问题。首先，单目SAM3D式的物体重建方法可能会产生错误的姿态和不可见的几何结构；而有效的操作则依赖于稳定且可校准的多视图数据。其次，原始的KnowledgeBank主要只是提取语义上相似的信息并添加新知识，这导致难以控制记忆的质量、冲突情况、置信度以及几何信息的相关性。为了解决第一个问题，我们引入了GeoFuse-MV3D这一基于几何先验的MV-SAM3D重建方法。该方法通过输入视图掩码来验证外部几何信息，采用软视觉轮廓支持机制，进行轴方向优化处理，同时只保留几何信息而不影响外观特征。为了应对第二个问题，我们将KnowledgeBank升级为一种有管理的长期记忆系统，该系统包含质量、置信度、生命周期、验证者等相关元数据，同时具备精确的检索功能。最后，我们在GSO-30任务中评估了重建模块的性能，在Terminal-Bench 2.0和SWE-Bench Verified任务中则评估了记忆模块的性能。GeoFuse-MV3D相比MV-SAM3D基线模型，将CD和LPIPS分别降低了2.20%和2.02%，同时PSNR和SSIM则分别提升了2.36%和1.03%；而KnowledgeBank则相比ReasoningBank在Terminal-Bench SR任务中提升了4.53%，在SWE-Bench的解析率方面提升了3.73%，同时AS值则分别降低了4.95%和5.65%。代码链接：https://github.com/AIGeeksGroup/GeneralVLA-2。网站地址：https://aigeeksgroup.github.io/GeneralVLA-2。
English Abstract
Generalist vision-language-action systems need object-centric 3D evidence and reusable manipulation experience to plan reliable robot trajectories. GeneralVLA provides a hierarchical interface for converting language and RGB-D observations into 3D end-effector paths, but two bottlenecks remain. First, monocular SAM3D-style object reconstruction can hallucinate pose and unseen geometry, while manipulation benefits from stable object shape when calibrated multi-view observations are available. Second, the original KnowledgeBank mainly retrieves semantically similar snippets and appends new knowledge, which makes it difficult to control memory quality, conflicts, confidence, and geometric relevance. To address the first challenge, we introduce GeoFuse-MV3D, a geometry-prior-guided MV-SAM3D reconstruction branch that verifies external geometry cues with input-view masks, applies soft visual-hull support, performs axis-wise refinement, and fuses only geometry while preserving appearance. To address the second challenge, we upgrade KnowledgeBank into a governed long-term memory system with explicit quality, confidence, lifecycle, verifier, and conflict metadata, together with precision-oriented retrieval. Finally, we evaluate the reconstruction branch on GSO-30 and the memory module on Terminal-Bench 2.0 and SWE-Bench Verified; GeoFuse-MV3D improves over the MV-SAM3D baseline by reducing CD and LPIPS by 2.20% and 2.02% while increasing PSNR and SSIM by 2.36% and 1.03%, and KnowledgeBank improves over ReasoningBank by 4.53% on Terminal-Bench SR and 3.73% on SWE-Bench resolve rate, while reducing AS by 4.95% and 5.65%, respectively. Code: https://github.com/AIGeeksGroup/GeneralVLA-2. Website: https://aigeeksgroup.github.io/GeneralVLA-2.