语义浏览:可控制的图像生成多样性
Semantic Browsing: Controllable Diversity for Image Generation
摘要
现代文本到图像模型在视觉逼真度和对提示的遵循性方面表现出色。然而,这种严格的遵循性却以多样性为代价:生成的样本往往只能呈现单一的视觉风格。现有的提高多样性的方法所产生的结果,更多是由偶然变化而非有意义的设计选择所驱动。这促使我们提出了一种新的多样性实现方式——即强制对生成样本进行结构化处理。我们提出了一种能够实现可控多样性的方法,使得用户能够浏览有结构的图像画廊,并通过系统化的方式探索各种有意义的、可理解的变异方式,从而进行创造性的探索。要实现这种语义层面的控制,就需要对场景有深入的理解。我们注意到,最近的文本到图像模型是在复杂的描述文本上训练的,因此语义决策与像素生成被有效分离了。这就带来了一种范式转变:我们不再依赖文本到图像模型内部的随机变化,而是直接在文本层面引入多样性。通过利用丰富的文本表示,我们可以让视觉语言模型能够完全理解整个场景的上下文。为了克服标准视觉语言模型通常产生的通用输出,我们采用了一种代理式工作流程,能够明确地实现与原始提示相一致的结构化变异。我们的方法能够产生多样化且易于导航的设计方案,每种变异都对应着一种具体、用户可理解的语义决策。
English Abstract
Modern text-to-image models excel in visual fidelity and prompt adherence. However, this strict adherence comes at the cost of diversity: generated samples tend to collapse into a single visual interpretation. Existing methods to improve diversity produce outputs driven by incidental variations rather than meaningful design choices. This motivates a new variant of the diversity task where structure is enforced on the generated samples. We introduce a method for controlled diversity that enables Semantic Browsing, where users can navigate structured image galleries and experience creative exploration through a systematic traversal of meaningful, interpretable axes of variation. Achieving this level of semantic control requires a deep understanding of the scene. We exploit the fact that recent text-to-image models are trained on elaborated captions, effectively decoupling semantic decision-making from pixel generation. This enables a paradigm shift: instead of relying on stochastic variation within the text-to-image model, we induce diversity directly at the text level. By leveraging rich textual representations, we allow a Vision Language Model (VLM) to operate on the full scene context. To overcome the generic outputs typical of standard VLMs, we employ an agentic workflow that explicitly enforces structured variation attuned to the original prompt. We demonstrate that our method produces diverse and navigable design spaces where every variation corresponds to a specific, user-understandable semantic decision.