提醒:通过重构来记住那些能够持续生成长视频的内容。
Memento: Reconstruct to Remember for Consistent Long Video Generation
摘要
长格式视频的生成要求那些反复出现的角色在不同的镜头、视角、动作以及场景转换中保持一致性。现有的时间分割方法通过逐帧生成视频来提升可扩展性。不过,这些方法主要专注于优化后续帧中的连贯性,而并未考虑如何确保历史记忆中保留与角色身份相关的信息。因此,随着生成的进行,这些反复出现的角色可能会被淡化、覆盖或遗忘。在本文中,我们提出了Memento这一框架——一种以角色保留为目标的重建方法。我们认为,如果一个记忆库能够忠实地保存某个角色的信息,那么就可以仅依靠该记忆库来重建这个角色的形象。具体来说,Memento结合了自回归式下一帧生成方法与基于记忆的角色重建技术,利用历史记忆和全局故事描述来恢复角色的形象。为了将长期存在的角色信息与短期线索区分开来,Memento采用了双查询机制:一个查询用于获取与角色身份相关的信息,另一个则用于选择适合连贯性描述的短文本帧。此外,一种注重角色细节的电影级数据处理流程,通过一致且无需使用代词的角色描述,为角色重建提供了精确的指导。实验表明,Memento在长期角色一致性、跨帧连贯性以及视觉质量方面取得了卓越的性能。
English Abstract
Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next-shot continuations without verifying whether the historical memory preserves identity-critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next-shot generation with memory-based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long-range subject evidence from short-range cues, Memento introduces a dual-query memory mechanism, where one query retrieves identity-relevant memory and the other selects short-context keyframes for coherent continuation. Additionally, a subject-aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun-free subject descriptions. Experiments demonstrate that Memento achieves state-of-the-art performance in long-term subject consistency, cross-shot coherence, and visual quality.