Refactor memory management

[muellerberndt]

The current memory code is pretty ad-hoc and unoptimized. It's unclear whether the query method and values used lead to optimal results in long-running tasks. Some questions:

• Is it better to summarize memory during storage or during retrieval (at the moment we use a mix of both)?
• Can the query be improved?
• Is 5 items a sensible number?
• What are the optimal settings for MAX_CONTEXT_SIZE and MEMORY_ITEM_SIZE for GPT-3.5 and GPT-4?
• Are there other ways to improve the memory system?

Ideally, several variants and tweaks should be debugged & tested on long-running tasks that require lots of memory, such as summarizing multiple websites or similar.

[Koalamanx]

I have been trying to get Micro-GPT working for 2 days. I cant sign up for Pinecone as is doesnt take sign ups atm. I can install Chromeadb as it crashes whilst installing on my Raspi 4.

Kind of bummed. What can I do?

[ismaelc]

From what I observed:

• Pulling the last X results (sliding window) - works great for objectives that require sequential actions like writing code projects into a folder
• Summarization - Didn't do that many tests here, but I imagine this is useful when results are being pulled from sources that require a big context window e.g. web results, reading lengthy files, etc.

[ismaelc]

I have been trying to get Micro-GPT working for 2 days. I cant sign up for Pinecone as is doesnt take sign ups atm. I can install Chromeadb as it crashes whilst installing on my Raspi 4.

Kind of bummed. What can I do?

Can you create an issue so we can track. Also paste the error you're getting when installing chromadb

[muellerberndt]

PR #37 should address this issue but still needs some work. Feel free to give your feedback on that PR. @ibrarzahoor @ismaelc

[muellerberndt]

I found a good prompt for testing the memory implementation:

python microgpt.py "Come up with an original idea for an online job that you can do autonomously. It is important that you can do this job with minimal support by the user. You must handle sourcing of clients, communication with clients, and payment by yourself. Proceed to find clients and perform the job. If you no need any support, such as creating an account, ask the user to help. All payments should go to the Paypal account [paypay account]."

This reliably triggers long-running tasks with many steps (and with pretty interesting results 🙂).

[muellerberndt]

Hey everyone, I just created a Telegram group for coordinating development. If you like to join, drop me a DM on Twitter for the link.