将示例转化为任务指令：提升现实世界B2B对话中的上下文内学习能力
Distilling Examples into Task Instructions: Enhanced In-Context Learning for Real-World B2B Conversations
摘要
上下文学习是一种用于资源匮乏场景下的分类问题的常用方法，但在特定领域中的效果仍未被充分研究。我们致力于解决对语义复杂、涉及多方的B2B对话进行分类的问题——传统上下文学习方法存在明显局限，尤其是当上下文长度因多个少量样本的串联而增加时。我们引入了Call Playbook数据集，该数据集包含五个基于真实B2B对话的分类任务，这些任务与核心销售概念相关。为了将性能与实际应用效果结合起来，我们提出了新的知识提取方法，能够将冗长的示例转化为简洁且易于理解的结构化分类标准和任务描述。我们的方法使得标记使用量减少了99%，同时宏观平均AUC值也提升了7%。值得注意的是，即使上下文长度增加，该方法仍然保持稳定，而一些先进的标记压缩方法则会导致F1分数下降超过9个百分点。重要的是，我们的框架能够直接优化分类逻辑，从而满足现实世界NLP应用中对透明度、效率以及用户交互方面的需求。
English Abstract
In-context learning (ICL) is the standard method for low-resource classification, yet its efficacy in specialized domains remains largely unexplored. We address the challenge of classifying semantically complex, multi-party B2B conversations, where traditional ICL encounters significant limitations, especially as context length increases due to the concatenation of multiple few-shot examples. We introduce the Call Playbook dataset, featuring five classification tasks derived from real-world B2B conversations targeting core sales concepts. To bridge the gap between performance and practical utility, we propose novel knowledge extraction methods that distill verbose examples into compact, interpretable representations of structured classification criteria and precise task descriptions. Our approach achieves a 99\% reduction in token usage and improves macro-averaged AUC by up to 7\% over traditional ICL. Notably, it remains robust as context grows, unlike advanced token compression baselines which degrade by over 9 F1 points. Importantly, our framework enables direct refinement of classification logic, addressing critical needs for transparency, efficiency, and user interaction in real-world NLP applications.