Memory postgresml

[santiatpml]

Background

PostgresML can be used as an memory backend for Auto-GPT. PostgresML is an open-source machine learning extension to PostgreSQL that enables training and inference on text and tabular data. With PostgresML, you can integrate machine learning models into your PostgreSQL database. In addition, you can use your existing PostgreSQL database as a vector database by generating embeddings from text stored in the database.

Resources:

• Generating LLM embeddings with open source models in PostgresML
• PostgresML as a Vector Database

Changes

• .env.template includes a section on PostgresML environment variables
• autogpt/config/config.py includes PostgresML configuration
• autogpt/memory/__init__.py includes PostgresMLMemory as a supported memory backend
• autogpt/memory/postgresml.py A new Python file and a new class called PostgresMLMemory developed using redis implementation as a reference

Documentation

• Each file that has been edited/added has respective inline comments
• Updated docs/configuration/memory.md with an additional section on PostgresML installation instructions and pip dependencies
• pip dependencies for PostgresML are not included in the requirements.txt

Test Plan

These steps are included in documentation. Rewriting summary here:

1. Install PostgresML using Docker or spin up a PostgresML cloud database
2. Clone Auto-GPT and run pip install -r requirements.txt
3. To use PostgresML memory backend:
   • Install libpq-dev : sudo apt install libpq-dev
   • pip install psycopg2
4. python -m autogpt -m postgresml will use PostgresML as memory backend

PR Quality Checklist

• [x] My pull request is atomic and focuses on a single change.
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

I have not added tests related to this PR as this involves additional dependencies like spinning up a PostgresML database and Python packages. I have not seen any tests for redis or pinecone for reference. Happy to add any suggested tests.

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[katmai]

any chance you can create a quality of life PR that would deal with taking redis data that would already be there, if we've run autogpt for a while now, and importing it seamlessly into PostgresML?

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)	Visit Preview		Apr 26, 2023 11:55pm

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[amarkules1]

just pulled your updates and everything seems to be working now, looks like you've got some merge conflicts to resolve though

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[p-i-]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

[ntindle]

Hey, I've marked this as don't merge until the Memory Fixes are in. Sorry to keep it on hold longer, just no real way to test/validate functionality until the fixes are in

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.