BRDFusion：物理原理与新一代城市场景反渲染技术的结合
BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering
摘要
从拍摄的视频中逆向渲染城市场景可以应用于多种场景，包括内容创作和自动驾驶模拟等。基于物理的渲染方法能够精确模拟光照效果，但会在渲染过程中产生一些伪影。而生成式模型则能够生成逼真视频，不过其一致性和可控性相对较差。我们提出了BRDFusion这一统一框架，该框架结合了两种互补的模型，用于逆向和正向渲染。具体来说，BRDFusion能够通过物理建模来恢复场景中各元素的精确属性，同时通过生成式模型来消除优化过程中的不确定性。在正向渲染过程中，物理模型能够根据场景配置来精确控制渲染过程，而生成式模型则能够去除渲染过程中产生的伪影。因此，我们的方法能够生成高质量的视频，同时又能实现精确的控制，其性能优于现有的基准算法，无论是在真实场景还是合成场景中都是如此。此外，BRDFusion还支持多视角重新照明、夜间模拟以及动态物体的插入与编辑等功能。项目页面：https://shigon255.github.io/brdfusion-page/
English Abstract
Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/