Building an AI Agent with Memory: Designing and Implementing a Three-Layer Memory Model

目录

Introduction: Why Does an AI Agent Need Memory?

When using large language models like ChatGPT and Claude, have you ever run into this frustration: every time you start a new conversation, the AI seems to have “amnesia,” remembering nothing of your previous exchanges. You have to repeat your background, preferences, and needs over and over. This “goldfish memory” severely limits AI’s ability to become a true personal assistant.

To let an AI Agent keep learning and growing like a human, I designed and implemented a three-layer memory model. This system draws inspiration from the human memory mechanism in cognitive science, letting the AI remember the user’s identity, preferences, important facts, and even long-term cross-session knowledge.

The Three-Layer Memory Architecture: Mimicking Human Cognition

The human memory system is divided into working memory (holding information short-term), short-term memory (recent events), and long-term memory (persistent knowledge). Borrowing this idea, and drawing on OpenClaw’s design, I reproduced an AI memory system with the following three layers:

Layer One: Working Memory

Working memory is the AI’s “temporary workbench,” storing the full context of the current conversation session. It’s like the focus of human consciousness, letting the AI understand and respond to the user’s current input.

Core features:

• Complete conversation record: keeps all user inputs and AI replies in the current session
• Token limit management: monitors context length to avoid exceeding the model’s processing limit
• Real-time access: the AI can immediately retrieve all information in the current session

Implementation example:

@dataclass
class WorkingMemory:
    messages: List[Dict[str, Any]] = field(default_factory=list)
    context_window: int = 4000  # token limit hint
    
    def add_message(self, role: str, content: str, metadata: Dict = None):
        """Add a message to working memory"""
        self.messages.append({
            "role": role,
            "content": content,
            "timestamp": datetime.now().isoformat(),
            "metadata": metadata or {}
        })

Layer Two: Short-term Memory

When working memory grows too large, short-term memory comes into play. It compresses old conversations into summaries and extracts key points, letting the AI handle long conversations efficiently without losing important information.

Core features:

• Conversation summarization: compresses lengthy conversations into concise summaries
• Key point extraction: identifies and saves the important information points in a conversation
• User intent tracking: records the user’s main goals and needs in the session

Use case: after you’ve had 50 rounds of in-depth technical discussion with the AI, it can still accurately recall the core issues you discussed, without having to reread the entire conversation.

Layer Three: Long-term Memory

Long-term memory is the essence of this system. It lets the AI remember the user’s information across sessions, forming a persistent user profile. This is the key to turning AI from a “tool” into a “partner.”

I designed seven memory categories:

Category	Icon	Description	Example
identity	👤	Identity information	“The user is named Awen and is a software engineer”
preference	⭐	Preferences	“The user likes concise answers”
fact	📌	Important facts	“The user is developing an AI Agent project”
lesson	📚	Lessons learned	“Last time there was a compatibility issue with Python 3.9”
goal	🎯	Goals and plans	“The user plans to launch the product within three months”
relationship	👥	Relationship information	“The user’s team members include Xiaoming and Xiaohong”
instruction	📜	Instructions	“Please always reply in Chinese”

Once the AI has learned this, when asked again it will answer with the remembered content based on historical conversations.

Intelligent Memory Extraction: Automatically Identifying Important Information

The coolest feature is automatic memory extraction. The system analyzes each round of conversation and automatically identifies information worth remembering, without the user having to state it explicitly.

Supported Natural-Language Patterns

1. Active memory requests

User: Please remember my email is awen@example.com
User: Remember, I hate replies that are too long

2. Identity information recognition

User: My name is Awen, and I'm a full-stack engineer
User: I work in Hangzhou

3. Preference extraction

User: I like concise and clear answers
User: I don't like using third-party libraries

4. Instruction capture

User: You should always give a code example first
User: Please don't use technical jargon

Implementation principle: using regular-expression pattern matching + LLM analysis to intelligently extract structured memories from conversations.

Memory Injection: Giving the AI Context Awareness

How do the extracted memories influence the AI’s replies? The answer is memory injection. Before each conversation begins, the system automatically injects the relevant memories into the system prompt:

==================================================
[Memories about the user]
User name: Awen
User traits: software engineer, full-stack developer
Communication style: concise and direct

👤 Identity information:
  • User name: Awen
  • User role: full-stack engineer

⭐ Preferences:
  • User likes: concise and clear answers
  • User dislikes: replies that are too long

📌 Important facts:
  • The user is developing a LangGraph Agent
  • The user uses Python 3.11
==================================================

Please use the above memory information to better understand and help the user.

This way, the AI takes the user’s background and preferences into account in every reply, achieving true personalized service.

Technical Implementation: Markdown Storage + Priority Management

Markdown-Format Storage

Long-term memory is stored in Markdown files, making it easy for humans to view and edit:

# ⭐ Preferences

## 2026-04-03 14:30 | Importance: ⭐⭐⭐

The user likes concise and clear answers, and dislikes lengthy explanations

> Source: session_a1b2c3d4

---

## 2026-04-02 09:15 | Importance: ⭐⭐⭐⭐

The user prefers to develop in Python, and doesn't like JavaScript

> Source: session_e5f6g7h8

---

Memory Priority

Not all memories are equally important. The system assigns each memory a 1–5 level of importance:

• ⭐⭐⭐⭐⭐ Identity information (highest priority)
• ⭐⭐⭐⭐ Instructions
• ⭐⭐⭐ Important facts, preferences
• ⭐⭐ Lessons learned
• ⭐ Temporary information

During memory injection, the system loads high-priority memories first, ensuring that the most critical information is retained within the limited context window.

Application Scenario: A Real Conversation Example

Let me demonstrate the practical effect of this memory system:

First conversation (new user)

User: Hello, my name is Awen, and I'm a software engineer
AI: Hello Awen! Nice to meet you. How can I help you?

[The system automatically extracts memory: user name = Awen, role = software engineer]

Second conversation (cross-session)

User: Help me take a look at what's wrong with this code
AI: Sure, Awen. As a software engineer, you should have high standards for code quality.
Let me analyze it for you...

[The AI automatically recalled the user's name and role]

Third conversation (deeper personalization)

User: I want to learn LangGraph
AI: Awen, considering you're a software engineer, I recommend starting with LangGraph's core concepts:
StateGraph, Nodes, and Edges.
Would you like me to provide a complete example project?

[The AI combined the user's identity to give targeted suggestions]

Integration with Other Technologies

Integration with LangGraph

def run_agent(user_input: str, thread_id: str = "default", user_id: str = None):
    # 1. Add to working memory
    memory_store.add_to_working_memory(thread_id, "user", user_input)
    
    # 2. Inject long-term memory into the system prompt
    memory_context = memory_store.get_system_prompt_with_memory(user_id)
    messages = [SystemMessage(content=memory_context)]
    messages.append(HumanMessage(content=user_input))
    
    # 3. Run the conversation
    result = graph.invoke({"messages": messages}, config=config)
    
    # 4. Extract new long-term memories
    memory_store.extract_memories_from_dialogue(user_id, session_id, user_input, ai_msg)
    
    return result

Collaboration with MCP Services

The memory system works seamlessly with MCP (Model Context Protocol) services. When the AI calls an external tool, the memory context is passed along, ensuring tool calls can also enjoy personalized service.

Future Outlook

This memory system still has a lot of room to evolve:

1. Memory vectorization: use a vector database to store memories, supporting semantic retrieval
2. Forgetting mechanism: simulate the human forgetting curve, automatically clearing outdated memories
3. Cross-user memory: identify commonalities between user groups and provide collective intelligence
4. Memory visualization: provide users with a memory management interface to view and edit what the AI remembers

Conclusion

Memory is the cornerstone of intelligence. Without memory, AI can only be a one-off tool; with memory, AI can become a truly intelligent partner that understands you and stays by your side.

This three-layer memory model is already running in my LangGraph Agent project, letting the AI assistant evolve from a “goldfish” into a “confidant.” If you’re also interested in building an AI with memory, you’re welcome to join the discussion!

---

Related tech stack:

• LangGraph - workflow orchestration
• LangChain - LLM application framework
• Python 3.11+ - development language
• Markdown - memory storage format

Reference projects:

• OpenClaw - an open-source memory-augmented AI assistant
• MemGPT - virtual context management

中英对照

文章作者：阿文

文章链接： https://www.awen.me/post/dfbce9ea.html

版权声明：本博客所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 阿文的博客！