Long-Term Memory

[t41372]

Long-Term Memory Implementation

Background

Long-term memory has been consistently among our most requested features. While we've made some attempts (mem0 and MemGPT), they weren't that great at the time and were removed during refactoring cycles. We welcome PRs that add them back.

The agent architecture introduced in v1.0.0 was specifically designed to support multiple long-term memory systems with the ability for users to switch between implementations of memory implementation. Therefore, despite that I have my idea about the long-term memory implementation in our project, we are open to different thoughts and implementations and welcomes PRs with different ideas.

Considerations

Here are some considerations when I think of long-term memory implementation in our project. They don't have to be strictly enforced by all of our long-term memory solutions, but they are definetely worth considering.

1. Performance: Fast and efficient retrieval of stored information.
2. Ability to remember both user information and the AI's response
3. Model compatibility: Work with existing models in our project

Current State

I plan to implement my version of long-term memory in the next version v1.2.0 (v1.1.0 is about to be released with many cool features). Meanwhile, thoughts about long-term memory can be discussed here and we are very welcome if anyone is interested in implementing one.

[ylxmf2005]

Brainstorm

• 对话内容记忆

  • 挑战
    • 时序问题
    • 如何存储？
    • 如何检索？

• 抽象记忆

  • 用户画像

    • 记录用户的显性的信息，类似图结构，与时间关联性较弱（不易随时间衰减）
      • 用户 姓名 xxx
      • 用户 身份 xxx
      • 用户 生日 xxx
      • 用户 宠物 xxx
      • …

  • Insight 事件 / 对话记忆

    • 高价值的事件
      • 用户和 AI 产生的对话有很高的惊喜值（比如 AI 或用户产生了一次有趣的互动）
      • 用户在对话中提到了重要的事件（比如下周要参加考试）
    • 事件具有的属性
      • 用于语义检索的 key
      • Insight (value)
      • 时间
      • 重要度 / 惊喜度

• 低负载和高负载两种模式

• 并行而非串行

Challenge

• 记忆自然化（避免：根据记录…）
• 避免记忆的过拟合
• 冲突和更新
• 遗忘机制、记忆可信度

Extra Features

• 管理记忆的可视化界面
• 多模态信息（主要是图片）用于记忆