text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

OpenAI extension list currently loaded models not dummy models.

Open jonnysowards opened this issue 11 months ago • 4 comments

Fixes bug #5675 that lists dummy models instead of loaded models in OpenAI extension. v1/models.

Checklist:

jonnysowards avatar Mar 18 '24 01:03 jonnysowards

What will this do to tools that expect openAI models or choose them?

Ph0rk0z avatar Mar 27 '24 12:03 Ph0rk0z

@Ph0rk0z That is one of my concerns. When using integrations like continue.dev the proper models do not load via the open ai api endpoints, only dummy ones that do not work. Although if another integration is required to have to have the open ai models to verify its working then this could cause it to fail. I'm wondering if potentially a flag could denote whether or not to use dummy models.

jonnysowards avatar Mar 28 '24 00:03 jonnysowards

Why not just add gpt4/3.5 fake model strings in addition to the actual models and make those not switch the current model? Kill two birds with one stone in a way.

Ph0rk0z avatar Apr 01 '24 16:04 Ph0rk0z

@Ph0rk0z That sounds like the best of both worlds, and easy to implement. I will update the pull to do that when I get a minute.

jonnysowards avatar Apr 01 '24 16:04 jonnysowards