FedOT：通过水印技术对所有权进行验证并追踪数据泄露情况，适用于联邦式LDM系统
FedOT: Ownership Verification and Leakage Tracing via Watermarks for Federated LDMs
摘要
在联邦学习框架下对潜在扩散模型进行训练，因为能够结合潜在扩散模型的强大生成能力与联邦学习的隐私保护特性，因此受到了越来越多的关注。不过，联邦学习需要让多个参与者共享全局模型，这容易导致模型被未经授权的客户端分发或转售。一种直观的做法是采用现有的基于VAE的水印技术来对潜在扩散模型进行水印处理，但这种方法存在两个根本问题：(1) 现有方法虽然可以验证模型的归属权，但无法追踪模型被泄露给特定恶意客户的情况；(2) 基于VAE的水印很容易被移除，只需用另一个干净的模型替换解码器即可。本文提出了FedOT框架，这是首个用于联邦式潜在扩散模型中验证归属权和追踪模型泄露情况的框架。具体来说，为了解决第一个问题，我们设计了分块式水印：第一部分用于验证模型的归属权，第二部分则用于识别具体的客户端。此外，为了应对第二个问题并保护模型免受VAE被替换的攻击，我们引入了潜在向量变换技术，通过修改VAE的原始潜在分布，从而增强VAE与U-Net潜在空间之间的联系。因此，任何试图用其他模型替换VAE以去除水印的行为都会导致图像质量严重下降，使得该潜在扩散模型无法继续使用。大量实验表明，FedOT在验证归属权和追踪模型泄露方面都表现出优异的性能。项目页面：https://spyzixuan.github.io/FedOT/。
English Abstract
Training Latent Diffusion Models (LDMs) within Federated Learning (FL) has attracted increasing attention due to its ability to combine the powerful generative capacity of LDMs with the privacy-preserving properties of FL. However, FL requires sharing the global model with multiple participants, which risks unauthorized model distribution or resale by malicious clients. While an intuitive approach is to adopt existing VAE-based watermarking techniques for LDMs in FL, this strategy falls short in addressing such threats due to two fundamental challenges: (1) Existing methods support ownership verification but lack the ability to trace model leakage to a specific malicious client; (2) VAE-based watermarks are vulnerable, as they can be removed simply by replacing the decoder with a clean counterpart. In this paper, we propose FedOT, the first framework for ownership verification and leakage tracing in federated LDMs. Specifically, to address the first challenge, we design a chunked watermark, where the first part is for ownership verification, and the second part is used for client identification. Furthermore, to overcome the second challenge and secure the model against VAE replacement attack, we introduce Latent Vector Transformation (LVT), which strengthens the connection between the VAE and U-Net latent spaces by modifying the original latent distribution of the VAE. Consequently, any attempt to replace the VAE for watermark removal leads to significant image quality degradation, making the LDM model unusable. Extensive experiments demonstrate that FedOT achieves superior performance in both ownership verification and traceability. Project page: https://spyzixuan.github.io/FedOT/.