FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

SillyTavern ui? FastChat compatible?

Open QuinnPiers opened this issue 1 year ago • 3 comments

Hi, sorry I'm a big newb at this.

I love that fastchat uses multi gpus and I've gotten the webservice/worker setup, but I don't know how to config SillyTavern to use fastchat and its not listed as one of the API connections options... https://docs.sillytavern.app/usage/api-connections/

but I wonder if Fastchat is the same as like Oobabooga? or 1 of the other options? https://docs.sillytavern.app/usage/api-connections/

QuinnPiers avatar Apr 20 '24 17:04 QuinnPiers

https://github.com/SillyTavern/SillyTavern/blob/1c9b89fdcceee5ebadd17e9b34f23eb9f0b37eab/src/endpoints/chats.js#L41

QuinnPiers avatar Apr 20 '24 17:04 QuinnPiers

Looks like fastchat has an openai interface, and sillytavern takes an openai interface... so it should work that way.

QuinnPiers avatar Apr 20 '24 17:04 QuinnPiers

so in {fastChat,serve.controller}/docs I see that the endpoint isn't /v1/models its "/list_models" which makes me wonder if FastChat is openai compatible?

QuinnPiers avatar Apr 20 '24 17:04 QuinnPiers