Vera：一种用于保留内容信息的视频编辑的多层扩散模型
Vera: A Layered Diffusion Model for Content-Preserving Video Editing
摘要
视频扩散模型在视频生成和编辑方面取得了显著进展。然而，内容保留仍然是一个核心挑战：现有方法会重新生成每一个像素，往往还会改变那些应该保持不变的元素，比如角色或背景场景。我们提出了Vera这一基于分层扩散技术的视频编辑框架，它并非重新生成整个视频，而是生成一个编辑层以及Alpha蒙版，以便与原始视频进行合成，从而将创意编辑与内容保留分开处理。为了促进与原始视频的协调合成，我们将文本到视频DiT技术扩展为混合Transformer架构，每个层都有独立的DiT模型，这些模型通过联合自注意力机制进行交互。为了支持Vera的训练，我们还构建了一个高质量的分层数据集，该数据集包含精确的Alpha蒙版、多种场景和动态效果等元素。在定量评估及人类偏好测试中，Vera在内容保留方面优于主流开源视频编辑模型，同时在编辑质量方面也具有竞争力，其训练数据包含486K帧的分层数据。
English Abstract
Video diffusion models have enabled remarkable progress in video generation and editing. However, content preservation remains a core challenge: existing methods regenerate every pixel and often alter elements that should remain unchanged, such as characters or background scenes. We introduce Vera, a layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design. To encourage coherent composition with the source video, we extend the text-to-video DiT into a Mixture-of-Transformers (MoT) architecture, with separate DiTs for each layer that interact through joint self-attention. To support the training of Vera, we further construct a high-quality layered dataset with accurate alpha mattes, diverse scenes and dynamics, and visual effects. Across our quantitative benchmark and human preference study, Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.