HiLo-Token：一种适应输入数据的高低频令牌压缩技术，用于高效的图像编辑
HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editing
摘要
创意图像编辑工具，如Photoshop中的“Remove”或“Generative Fill”功能，是用户日常使用中的核心工具，占据了Photoshop和Lightroom中流量的很大比例。然而，当前的生成式AI模型存在严重的延迟问题，当从基于卷积的U-Nets模型转换为扩散变换器模型时，这一问题更为明显。在我们对数百个具有不同掩模比例的示例图像进行的测试中，即使将模型从50步缩小到8步，仅DiT模块就占到了总模型延迟的73%。为了应对这一挑战，我们提出了HiLo-Token技术——一种输入自适应令牌压缩框架。该框架将更多的令牌分配给高频、上下文丰富的区域，而较少的令牌则用于低频区域。具体来说，对于用户指定的编辑区域，我们会保留所有属于膨胀掩模的令牌，以保持强烈的局部关联性和上下文相关性。在编辑区域之外，我们采用一种简单但有效的高频令牌选择策略来捕捉重要的局部细节，同时利用16倍下采样后的图像中的令牌来表示低频成分，从而保留模糊但全局性的结构。在实际评估数据上的大量实验表明，所提出的方法非常有效：在A100-80GB设备上，针对小、中、大掩模比例的图像编辑任务，DiT速度分别提高了3.13倍、2.59倍和1.67倍，而平均掩模比例分别为6.38%、15.92%和35.36%，且生成质量没有下降。
English Abstract
Creative image editing tools, such as Photoshop's Remove or Generative Fill buttons, are central to everyday customer use and account for a major share of traffic in Photoshop and Lightroom. However, current generative AI models face significant latency challenges, which become even more pronounced when transitioning from convolution-based U-Nets to Diffusion Transformers (DiTs). In our evaluation on hundreds of representative image editing samples spanning a wide range of mask ratios, the DiT module alone accounts for an average of 73% of the total model latency, even after being distilled from 50 timesteps down to 8 timesteps. To tackle this challenge, we propose HiLo-Token, an input-adaptive token compression framework that allocates more token budget to high-frequency, rich-context regions while assigning fewer tokens to low-frequency areas. Specifically, for the editing region specified by the user mask, we retain all tokens within a dilated mask to preserve strong locality and contextual relevance. Outside the editing region, we introduce a simple yet effective high-frequency token selection strategy based on spatial frequency to capture important local details, while using tokens from a 16x downsampled image to represent low-frequency components and preserve the blurry but global structure. Extensive experiments on production-level evaluation data validate the effectiveness of the proposed method, achieving 3.13x, 2.59x, and 1.67x DiT speedups on A100-80GB for image editing tasks across small, medium, and large mask ratio categories with average ratios of 6.38%, 15.92%, and 35.36%, respectively, without any regression in generation quality.