提示级蒸馏：一种非参数化的替代方法，用于实现高效的推理能力提升
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning
摘要
“Prompt-Level Distillation”技术能够从教师模型中提取推理模式，从而提升学生模型的性能，同时保持模型的可解释性并降低推理的延迟。该技术由Qwen/Qwen2.5-Coder-32B-Instruct实现。通常情况下，复杂的推理需要采用“思维链提示”方式来实现，但这种方法的准确性虽然高，但会导致极高的延迟以及较高的推理成本。另一种方法是微调较小的模型，但这往往会导致模型的可解释性下降，同时还会增加计算资源和操作开销。为了克服这些限制，我们提出了“Prompt-Level Distillation”技术。该技术从教师模型中提取明确的推理模式，并将它们整理成结构化的指令列表，供学生模型使用。通过Gemma-3 4B模型进行评估，我们发现“Prompt-Level Distillation”技术在StereoSet数据集上的F1分数从57%提升到了90%，在Contract-NLI数据集上的F1分数则从67%提升到了83%。此外，该技术的LogiQA准确率也提升到了70%。在Mistral Small 3.1数据集上进行的测试表明，该技术具有跨架构的通用性，使得这些小型模型能够在较低的延迟下达到与先进模型相当的性能。这些具有表达力的指令使得决策过程更加透明，从而可以完全通过人工方式验证逻辑的正确性。因此，这种技术非常适合法律、金融、内容审核等需要严格监管的行业，同时也适用于高并发场景和边缘设备。
English Abstract
Prompt-Level Distillation extracts reasoning patterns from teacher models to enhance student model performance while maintaining interpretability and reducing latency. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Advanced reasoning typically requires Chain-of-Thought prompting , which is accurate but incurs prohibitive latency and substantial test-time inference costs. The standard alternative, fine-tuning smaller models, often sacrifices interpretability while introducing significant resource and operational overhead. To address these limitations, we introduce Prompt-Level Distillation (PLD). We extract explicit reasoning patterns from a Teacher model and organize them into a structured list of expressive instructions for the Student model 's System Prompt . Evaluated using Gemma-3 4B, PLD improved Macro F1 scores on StereoSet (57\% to 90.0\%) and Contract-NLI (67\% to 83\%), while increasing LogiQA accuracy to 70\%. Similar results on Mistral Small 3.1 demonstrate cross-architecture generalizability , enabling these compact models to match frontier performance with negligible latency overhead. These expressive instructions render the decision-making process transparent, allowing for full human verification of logic, making this approach ideal for regulated industries such as law, finance, and content moderation, as well as high-volume use cases and edge devices.