理解具有环境感知能力的信息检索行为
Understanding the Behaviors of Environment-aware Information Retrieval
摘要
最近的基于检索增强的生成技术已经证明了其在处理复杂查询方面的强大能力。然而，当前的研究却忽视了另一个关键问题：不同的检索器需要不同的查询构建策略才能获得最佳性能。在本研究中，我们首次系统地分析了大型语言模型如何通过强化学习来学会根据不同检索器的特性调整其查询构建策略。我们的实证研究表明，强化学习能够有效帮助大型语言模型根据特定检索器的特点来优化其查询方式。我们发现，不同的检索器具有截然不同的最优查询模式（例如，描述性查询与问题式查询），这表明针对一种检索器学到的策略在另一种检索器上可能并不有效。此外，我们还发现，通过结合针对各个检索器的具体指导以及扩大模型规模，可以进一步提升系统性能。为了促进多检索器阶段的训练过程，我们提出了一种基于分支的测试技术，从而改善训练的稳定性。我们的研究为构建真正考虑各种检索器的RAG系统提供了第一手的实证证据和实用建议。代码与资源可访问于https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval。
English Abstract
Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.