我们准备好迎接一种基于代理的记忆系统了吗？
Are We Ready For An Agent-Native Memory System?
摘要
大型语言模型代理的记忆机制已经从简单的检索增强机制迅速发展为一种能够支持持久性信息存储、检索、更新、整合以及动态生命周期管理的数据管理系统。尽管有了这样的发展，现有的评估方法仍然主要依据端到端的任务成功指标（如F1、BLEU等）来评价代理记忆功能，而把相关系统视为一个整体性的黑箱。因此，一些重要的系统层面问题，比如运营成本、不同记忆模块之间的架构权衡，以及面对动态知识更新时的鲁棒性等问题，仍然没有得到充分研究。在本文中，我们从数据管理的角度对代理记忆进行了系统的实验研究。我们提出了一种分析框架，将代理记忆分解为四个核心模块：记忆表示与存储、提取、检索与路由、维护。根据这一框架，我们对12种典型的记忆系统和2种参考基准在5种测试任务中进行了评估，这些任务涉及11个不同的数据集。我们的全面评估表明，没有任何一种架构能在所有情况下都表现出最佳性能；实际上，其有效性取决于记忆结构如何与任务中的瓶颈问题相契合。此外，通过详细的消融研究，我们量化了它们对表示精度、检索准确性、更新正确性以及长期稳定性各自的影响。最后，我们揭示了在真实工作负载下的成本与性能权衡情况，表明局部化的维护方式比全局性的重组更经济高效。基于这些发现，我们指出了构建真正适合代理使用的记忆系统的有前景的方向。相关代码可在https://github.com/OpenDataBox/MemoryData上获取。
English Abstract
Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.