EvoEmbedding：用于长上下文检索与智能体式记忆的可进化表示方法
EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory
摘要
现有的嵌入模型本质上是静态的：它们将文本片段单独编码，忽略了其周围的上下文和时间顺序。本文介绍了EvoEmbedding这一新型嵌入模型，它能够生成可进化的表示形式用于检索任务。该模型特别适用于长上下文场景，因为在这种场景中信息是动态的、具有序列性，需要持续跟踪状态。我们的设计很简单：EvoEmbedding在逐次处理输入数据时，会维护一个不断更新的潜在记忆机制，并将其与原始内容结合使用，从而生成可进化的嵌入表示。因此，对于相同的查询，我们的模型能够根据不断变化的上下文调整其表示方式，从而实现超越静态语义搜索的检索效果。为了让模型具备这种能力，我们构建了EvoTrain-180K这一多样化的数据集，用于潜在记忆与检索功能的联合优化。此外，我们还引入了记忆队列机制，以防止在重复编码过程中表示形式的崩溃，同时采用了分段批量处理技术来应对长度上的差异，从而将训练速度提升了3.8倍。大量实验表明，我们的模型不仅在各种长上下文检索测试中优于那些规模较大的专用模型（如Qwen3-Embedding-8B和KaLM-Embedding-Gemma3-12B），而且还能很好地应用于其他下游任务中，即使上下文长度是其训练窗口的10倍。值得注意的是，EvoEmbedding可以无缝集成到智能体工作流程中，从而提升性能。例如，使用我们模型的简单RAG流程就能超越那些专门的智能体记忆系统。项目页面：https://clare-nie.github.io/EvoEmbedding。
English Abstract
Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generates evolvable representations for retrieval. It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updated latent memory as it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain-180K, a diverse dataset for the joint optimization of latent memory and retrieval. Furthermore, we introduce a memory queue to prevent representation collapse during recurrent encoding, alongside segment-batching techniques that tackle significant length variance and accelerate training by 3.8times. Extensive experiments show that our model not only outperforms larger-scale specialists (e.g., Qwen3-Embedding-8B and KaLM-Embedding-Gemma3-12B) across a range of long-context retrieval benchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10times longer than its training window. Notably, EvoEmbedding seamlessly integrates into agentic workflows to boost performance. For instance, a naive RAG pipeline equipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare-nie.github.io/EvoEmbedding.