Add FastChat API for local models.

[BillSchumacher]

This adds a working dockerized setup for FastChat local models. They appear to use an OpenAI compliant URL scheme:

Background

Changes

Documentation

Test Plan

PR Quality Checklist

• [ ] My pull request is atomic and focuses on a single change.
• [ ] I have thoroughly tested my changes with multiple different prompts.
• [ ] I have considered potential risks and mitigations for my changes.
• [ ] I have documented my changes clearly and comprehensively.
• [ ] I have not snuck in any "extra" small tweaks changes

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			May 2, 2023 0:11am

[BillSchumacher]

Need to add config options, add chat completion call and documentation.

[codecov[bot]]

Codecov Report

Patch and project coverage have no change.

Comparison is base (4767fe6) 60.55% compared to head (f7d938d) 60.55%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3651   +/-   ##
=======================================
  Coverage   60.55%   60.55%           
=======================================
  Files          69       69           
  Lines        3184     3184           
  Branches      528      528           
=======================================
  Hits         1928     1928           
  Misses       1122     1122           
  Partials      134      134           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[BillSchumacher]

docker compose up -f docker-compose-fastchat.yml --build --wait

[ntindle]

Based on discussion with @Torantulino we should expand the plug-in interface to support this rather than putting them in the main repo.

I see multiple categories of plugins coming to exist with some of the obvious being:

• Models
• Memory
• Commands

[BillSchumacher]

I was just going to monkey patch the openai url.

[BillSchumacher]

Whoever thinks they can make dockerfiles into a plugin feel free.

[Boostrix]

Whoever thinks they can make dockerfiles into a plugin feel free.

sorry, asking for clarification: are you asking for a plugin to generate docker files ?

[BillSchumacher]

Whoever thinks they can make dockerfiles into a plugin feel free.

sorry, asking for clarification: are you asking for a plugin to generate docker files ?

No, this PR is just dockerfiles ATM. I was going to finish implementing this as part of the core, which made a lot of sense to me as it would require minimal code changes and is commercially licensable. I might do that on my fork instead.

[micelvrice]

hello, what should i do to use FastChat API for AutoGPT

[p-i-]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

[BillSchumacher]

hello, what should i do to use FastChat API for AutoGPT

You can see how I did this in my fork.

[Pwuts]

Since #2594, you can set a custom backend URL through the OPENAI_API_BASE_URL env/config variable. This + a container running FastChat should suffice, right?

I think we should be able to add extra docker-compose files with configurations like this. Not sure whether to put those in the project root or some other folder.

[ntindle]

Yeah ideally this would exist as a dedicated docker compose, but that may be a bit much for normal users. I'm growing more in more in favor of setup scripts