FreeStyle：通过社区LoRA挖掘技术实现对风格与内容双重参考的生成方式的自由控制
FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Mining
摘要
风格与内容双重参考生成技术旨在创造出一种图像，该图像既保留了内容信息的结构和含义，又具有独立风格参考所呈现的风格特征。尽管近年来取得了一定进展，但这一任务仍然具有挑战性，因为模型需要平衡内容准确性、风格一致性以及对指令的遵循程度，同时避免风格参考带来的语义泄露问题。一个主要的障碍是缺乏大规模的数据集，这些数据集能够清晰区分内容与风格，并且涵盖多种不同的风格类型。在本研究中，我们提出了FreeStyle这一可扩展的双重参考生成框架，该框架基于社区LoRA技术构建。我们将社区LoRAs视为风格与内容的组合基础，并设计了一套严格的生成与筛选流程，以在多个基础模型中生成大规模的风格参考与内容参考三元组。为了解决内容泄露问题，我们采用了两阶段课程式训练方法，并引入特定阶段的分离机制：在风格转换阶段，通过注意力层面的约束来抑制风格参考的泄露；而在复杂的双重参考阶段，则采用基于频率感知的RoPE调制策略来消除基于位置对应的泄露现象。我们还创建了一个包含风格参考和双重参考生成的评估基准，该基准从风格相似性、内容保留性、美学效果、对指令的遵循程度以及泄露抑制等方面进行评估。该基准还包含了一种与风格无关的内容对齐评分指标，以及基于VLM的拒绝评分，用于评估生成的可靠性及泄露抑制效果。大量实验表明，我们的模型能够在风格一致性、内容保留性和泄露抑制方面实现良好的平衡。
English Abstract
Style-content dual-reference generation aims to synthesize an image that preserves the structure and semantics of a content reference while adopting the style of a separate style reference.Despite recent progress, this setting remains challenging because models must balance content fidelity, style alignment, and instruction following avoiding semantic leakage from the style reference.A key bottleneck is the lack of large-scale triplet data with clean content-style separation and broad long-tail style coverage.In this work, we propose FreeStyle, a scalable dual-reference generation framework based on community LoRA mining.We treat community LoRAs as compositional anchors for style and content, and design a rigorous generation and filtering pipeline to construct large-scale Style-Reference and Content-Reference triplets across multiple base models.To address content leakage, we adopt a two-stage curriculum with stage-specific disentanglement mechanisms: an attention-level enrichment constraint that suppresses style-reference leakage in the style-transfer stage, and a frequency-aware RoPE modulation strategy that targets positional-correspondence-based leakage in the harder dual-reference stage.We also introduce a benchmark covering both style-reference and dual-reference generation, with evaluations on style similarity, content preservation, aesthetics, instruction following, and leakage rejection. The benchmark incorporates a style-invariant Content Alignment Score (CAS) and introduces a calibrated VLM-based Rejection Score for evaluating generation reliability and leakage suppression.Extensive experiments show that our model achieves a strong balance among style alignment, content preservation, and leakage suppression.