对机器文本检测器的攻击仍然保留着特定的风格特征
Attacks on Machine-Text Detectors Retain Stylistic Fingerprints
摘要
尽管已经采用了各种逃避检测的技术，但机器文本的检测仍然具有挑战性。不过，当从多个文档中分析文本时，风格特征可以发挥重要作用，从而有效防御机器文本的攻击。这些文本是由Qwen/Qwen2.5-Coder-32B-Instruct生成的。虽然机器文本检测器的开发取得了显著进展，但机器文本很容易被设计成能够逃避检测的格式，这让人认为这个问题本质上是无法解决的。在这项研究中，我们探讨了这种逃避策略的局限性。我们发现，虽然当前的攻击手段——从简单的模式工程到利用检测器进行优化——能够有效降低标准检测器的性能，但这些攻击无法消除机器文本背后的风格特征。我们证明，那些利用风格特征空间的检测方法对于这类逃避攻击非常有效，即使面对经过特殊设计的模型所生成的样本，也能准确检测出来。这引发了这样一个问题：风格是否可以作为对抗机器文本检测攻击的通用防御手段呢？我们通过引入一种新的 paraphrasing方法来回答这个问题，该方法同时考虑了不可检测性和对特定人类风格的遵循性。结果表明，与之前的方法不同，这种攻击方式能够成功逃避所有检测器的检测，包括那些利用写作风格进行检测的模型。不过，我们发现这种逃避手段并非绝对有效的：随着可用于分析的文档数量的增加，人类文本和机器文本之间的区别会逐渐消失。总的来说，我们的研究结果表明，要实现可靠的机器文本检测，就需要从单文档分析转向多文档分析。
English Abstract
Machine-text detection remains challenging despite evasion techniques, but stylistic features can provide robust defense when analyzed across multiple documents rather than individual instances. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Despite considerable progress in the development of machine-text detectors , the ease with which machine-text can be manipulated to evade detection has led to suggestions that the problem is inherently intractable. In this work, we investigate the limits of such evasion strategies . We demonstrate that while current attacks, ranging from prompt engineering to detector-guided optimization can effectively degrade performance of standard detectors, they fail to erase the underlying stylistic "fingerprints" of machine text. We show that few-shot detectors that utilize the stylistic feature space are robust to these evasion attempts, reliably detecting samples even from models explicitly tuned to prevent detection. This raises the question: does style represent a universal defense against machine-detection attacks? We demonstrate that the answer is "no'' by introducing a novel paraphrasing approach that simultaneously optimizes for undetectability and adherence to specific human styles. We show that unlike prior methods, this attack effectively evades all considered detectors, including those that utilize writing style. However, we find that this evasion is not absolute: as the number of documents available for analysis grows, the human and machine distributions become distinguishable again. Overall, our findings suggest that reliable machine-text detection requires moving beyond single-document analysis to multi-document analysis .