How to use with LightLLM proxy

[fellnerse]

I would like to use sgpt with LightLLM proxy, but I only get this error:

Error: error, status code: 404, message: %!s(<nil>)

Any idea on what I'm doing wrong? The config is this:

maxtokens: 2048
model: bedrock-claude-v3.7
temperature: "1"
topp: "1"

[ChaosEternal]

You might need to set API_BASE_URL to your litellm proxy. And the model string should be bedrock/claude-...

[brandon-fryslie]

This does work btw, I've done it. But it requires the proxy to be running in many cases because right now sgpt does not support passing in args to configure LiteLLM nor any way (I can find) to set a path to the LiteLLM config file. The CONFIG_FILE_PATH argument that LiteLLM Proxy uses does not seem to be honored by the SDK.