genkit-plugins icon indicating copy to clipboard operation
genkit-plugins copied to clipboard

Support for openai baseurl/models

Open babybirdprd opened this issue 1 year ago • 5 comments

Is your feature request related to a problem? Please describe. No, it's adding to the open ai extension to allow for baseurl and model to be set in environment variables

Describe the solution you'd like A way to extend the function to open ai compatible endpoints

Describe alternatives you've considered Making it as it's own extension, as there are a lot of models that are not multimodal, or that at least don't have all the features of open ai. Although there are several projects that use the open ai api, to replace all features. Tts, stt, image generation, etc.

Additional context This would allow people that don't have access to claude, gemini, etc. The ability to run any llm through: openrouter, deepinfra, etc

babybirdprd avatar May 26 '24 18:05 babybirdprd

Thanks for opening the issue. I'm not familiar with those services, but after a brief look I think adding support should be quite easy. I'll look into it 👍🏻

Dabolus avatar May 28 '24 06:05 Dabolus

Thank you! Greatly appreciate it. And just as as an after thought. If you look at https://github.com/matatonic , they have a few different openai compatible API's for vision, stt, tts, and image gen. But that is more of a personal want, not sure if those integrations are as straight forward.

babybirdprd avatar May 30 '24 05:05 babybirdprd

Any updates on this feature?

ehelbig1 avatar Jul 12 '24 19:07 ehelbig1

It looks like this is already supported for Groq

ehelbig1 avatar Jul 12 '24 19:07 ehelbig1

Hi, I was hoping for the same feature for the OpenAI + Genkit, I started working on my own plugin when I found this one, so instead of working on everything again I opened up the PR #143, improving the config so we could add custom models and set the Base URL as well. It would be a really nice feature since we have a lot of services available with OpenAI API support (deepinfra, openrouter, litellm, etc)

MagdielCAS avatar Sep 08 '24 00:09 MagdielCAS

I proposed a fix for baseUrl support in #168

alexastrum avatar Nov 23 '24 04:11 alexastrum

close as it is done

xavidop avatar Feb 14 '25 08:02 xavidop