Featured image of post Agent Memory Evolution: From Simple Truncation to Vector Retrieval for Long-term Memory
AI News

Agent Memory Evolution: From Simple Truncation to Vector Retrieval for Long-term Memory

How LangChain Agents build long-term memory via summarization and Milvus

New post

1. Core Event: LangChain Agent Memory Upgrade

The author introduces two summarization strategies — message-count-triggered and token-count-triggered — combined with Milvus vector database for semantic retrieval, addressing the critical flaw of key information loss in traditional truncation.

  • Core components: @langchain/openai, @zilliz/milvus2-sdk-node, js-tiktoken
  • Key capabilities: LLM summarization compression + vector storage + semantic retrieval
  • Availability: Code is open-sourced with full reproducibility

2. The Fatal Flaw of Truncation and the Summarization Approach

Traditional context truncation (e.g., slice(-4)) directly discards old messages, causing permanent loss of important context — for example, user identity “Li Si” and profession “designer” are erased, resulting in model amnesia.

The core idea of summarization compression is to not directly discard old messages, but to compress them into a summary using LLM, then combine with recent messages. Eight original messages can be compressed into three (summary + latest 2 messages), with key information like name, profession, and skills preserved.

Two triggering strategies:

  1. Message count threshold: Trigger summary when exceeding maxMessages (e.g., 6), keep keepRecent (e.g., 2) most recent messages
  2. Token budget control: Use js-tiktoken for precise token calculation, dynamically retain recent messages based on token budget (e.g., keepRecentTokens = 80)

A key counterintuitive aspect: a single message may be just a few tokens (e.g., ‘I’m Li Si’) or hundreds (e.g., a detailed paragraph), making fixed-message truncation unable to precisely control context length, while token-budget strategy dynamically adapts to varying message token densities.

3. Technical Implementation and Vector Retrieval Loop

Summarization Function

  • getBufferString(): Converts Message array to readable format (“User: xxx / Assistant: xxx”)
  • SystemMessage injects summary prompt, calls LLM to generate summary
  • history.clear() +重组: Clear history, add back recent messages + summary

Vector Retrieval Architecture

  • Milvus: Open-source vector database supporting Docker standalone deployment
  • Embedding flow: Conversation text → vectorization → store in Milvus → semantic retrieval by similarity
  • Core capability: Breaks token limit, enables semantic-level long-term memory

Project structure includes 8 test files covering the complete pipeline from memory, file persistence, truncation, summarization (two strategies), Milvus insertion to semantic retrieval.

4. Adoption Recommendations and Use Cases

Ready to try now:

  • LangChain Agent developers needing long conversation context management
  • Applications requiring token cost control without losing historical key information

Consider waiting if:

  • Your team lacks Milvus operational capability: vector database deployment/maintenance requires extra infrastructure effort
  • Ultra-low latency is critical: semantic retrieval adds vector computation overhead

5. Final Thoughts

LangChain’s memory approach is evolving from “selective amnesia” to “permanent memory”. When context window becomes the capability ceiling, intelligently managing historical information matters more than simply expanding the window — this is an essential step toward practical agent systems.