世界模型中的幻觉是可预测且可预防的
Hallucination in World Models is Predictable and Preventable
摘要
现代生成式世界模型能够呈现越来越逼真的可控制未来场景，但往往会出现幻觉现象：虽然这些场景在视觉上看起来很自然，但实际上与真实情况存在偏差。我们认为，幻觉现象主要出现在状态-动作空间的低覆盖区域，而基于数据的轻量级信号可以帮助识别并缓解这种现象。为了验证这一假设，我们开发了MMBench2数据集——该数据集包含427小时、210个任务，包含真实动作、奖励以及实时模拟器，我们还在该数据集上训练了一个拥有350M参数的世界模型。我们发现了三种不同的幻觉模式：感知性幻觉、动作边缘化幻觉以及场景偏离幻觉——每种模式都对应着模型处理过程中的不同阶段。我们还提出了三种能够准确预测模型可能出现问题的信号。为了弥补训练过程中存在的覆盖不足问题，我们开发了一种基于覆盖意识的采样技术；而为了在线解决这一问题，我们的幻觉预测信号可以被用作目标数据收集的激励因素，从而实现了一种高效的数据优化方法，使得预训练的模型能够适应完全陌生的环境，而所需的实际环境数据量仅为50条轨迹。总体而言，我们的研究结果表明，世界模型中的幻觉现象本质上是由数据覆盖不足导致的，而用于检测幻觉的信号也可以被用来缓解这种现象。 我们论文的互动式网络版本可以在https://www.nicklashansen.com/mmbench2获取。
English Abstract
Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates in low-coverage regions of the state-action space, where lightweight data-centric signals can both detect it and guide mitigation. To test this, we introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators, and train a 350M-parameter world model on it. We identify three distinct hallucination modes: perceptual, action-marginalized, and scene-diverging -- each anchored to a different stage of the pipeline, and develop three signals that accurately predict where the model will fail. To close coverage gaps at training time, we develop a coverage-aware sampling technique; to close them online, our hallucination predictors serve as curiosity rewards for targeted data collection, yielding a data-efficient finetuning recipe that adapts the pretrained world model to entirely unseen environments with as few as 50 real environment trajectories. Overall, our findings reveal that hallucination in world models is inherently a data coverage issue, and that the same signals used to detect it can also be used for mitigation. An interactive web version of our paper is available at https://www.nicklashansen.com/mmbench2