Holo-World：用于视频世界模型的统一相机、物体和天气控制系统
Holo-World: Unified Camera, Object and Weather Control for Video World Model
摘要
视频世界模型正在朝着在可控的相机和物体运动条件下保留所观察到的世界状态的方向发展，同时允许其环境状态发生变化。不过，这些控制机制仍然处于孤立状态，而天气场景的生成通常依赖于已经明确指定了未来结构的源视频或重建场景。我们研究了一种以第一帧作为基准的源到状态生成方法：该模型从单张图像开始，遵循明确的相机和物体控制规则以及可选的天气指令，从而生成能够保留源世界状态或将其转移到目标天气状态的视频。为了应对这些挑战，我们首先构建了HoloStateData这一状态视频数据集，它将各种视频转化为可用于控制相机、物体和天气的统一样本。其次，我们提出了Holo-World这一统一的可控视频世界模型，它可以从单张图像出发来共同控制整个场景。该模型将世界状态的保留与天气状态的转移分别处理为不同的参数子空间，利用渲染出的背景、几何信息以及物体控制机制来保持可控的场景结构，同时模拟受天气影响的外观和粒子效果。此外，Scene-Weather Decomposed CFG则分别控制场景和天气的残余部分，从而增强目标天气效果，而不会过度放大整体条件。定量和定性实验表明，Holo-World能够在保持一致的场景结构的同时实现精确的相机和物体控制，同时能够将场景转移到不同的目标天气状态中，其性能优于传统的视频到视频天气编辑方法。我们的项目页面可在https://xiangchenyin.github.io/Holo-World/查看。
English Abstract
Video world models are moving toward preserving an observed world under controllable camera and object motion while allowing its environmental state to change. Yet these controls remain isolated, and weather generation typically relies on a source video or reconstructed scene that already specifies future structure. We study a first-frame-anchored source-to-state setting, where the model starts from a single image and follows explicit camera and object controls and an optional weather instruction, then generates a video that either preserves the source world or transfers it to a target weather state. To address these challenges, we first build HoloStateData, a state video dataset that turns diverse videos into unified control samples for camera, object, and weather supervision. Second, we introduce Holo-World, a unified controllable video world model that jointly controls scene from a single image. Its Unified Scene Adapter factorizes world preservation and weather transfer into distinct parameter subspaces, using rendered background, geometry buffers, and object controls to maintain controlled scene structure while modeling weather-dependent appearance and particle effects. Additionally, Scene-Weather Decomposed CFG guides scene and weather residuals separately, strengthening target weather effects without over-amplifying the full condition. Quantitative and qualitative experiments demonstrate that Holo-World maintains precise camera and object control with consistent scene structure while transferring scenes into diverse target weather state, outperforming video-to-video weather editing baselines on weather-state generation. Our project page is available at https://xiangchenyin.github.io/Holo-World/.