NextChat icon indicating copy to clipboard operation
NextChat copied to clipboard

[Bug] CUSTOM_MODELS Does not work when I upgraded to v2.11.3

Open weifeng12x opened this issue 11 months ago • 9 comments

Bug Description

When I upgraded to v2.11.3 from v2.9.x I found that CUSTOM_MODELS does not work, it worked on v2.9.x. My configuration is

  • CUSTOM_MODELS=-all,+gpt-3.5-turbo-0125,+gpt-4-0125-preview

Steps to Reproduce

  1. Set the environment in docker-compose.yml
  2. Chech the model list on client.

Expected Behavior

Its supposed to show 2 models which I listed in configuration file. However, it listed all models on client.

Screenshots

Configuration: image

Client model list: image

Deployment Method

  • [X] Docker
  • [ ] Vercel
  • [ ] Server

Desktop OS

No response

Desktop Browser

No response

Desktop Browser Version

No response

Smartphone Device

No response

Smartphone OS

No response

Smartphone Browser

No response

Smartphone Browser Version

No response

Additional Logs

No response

weifeng12x avatar Mar 18 '24 09:03 weifeng12x

try to open in InPrivate mode. Check whether it works.

Kosette avatar Mar 19 '24 15:03 Kosette

try to open in InPrivate mode. Check whether it works.

I tried different broswer, it still doesn't work. Addtional, not only i can see the model list, but also can use it if the api key have permission.

weifeng12x avatar Mar 19 '24 21:03 weifeng12x

Is it possible to use Claude 3 as a custom model?

steven-pribilinskiy avatar Mar 20 '24 18:03 steven-pribilinskiy

Likely related issue, even after adding the ANTHROPIC_API_KEY the Calude-3 models don't show up in model the list. Using: docker

crbon avatar Apr 17 '24 23:04 crbon

Likely related issue, even after adding the ANTHROPIC_API_KEY the Calude-3 models don't show up in model the list.

how did you start the container,docker run or docker compose? Have you recreate the container after changing config?

Add ANTHROPIC_API_KEY and CUSTOM_MODELS(if you using), recreate(not restart) container, then refresh in browser.

Kosette avatar Apr 18 '24 04:04 Kosette

Likely related issue, even after adding the ANTHROPIC_API_KEY the Calude-3 models don't show up in model the list.

how did you start the container,docker run or docker compose? Have you recreate the container after changing config?

Add ANTHROPIC_API_KEY and CUSTOM_MODELS(if you using), recreate(not restart) container, then refresh in browser.

Sorry, after adding the CUSTOM_MODELS environment variable, yes those custom models come through. Although, I'm still not clear on how to add Anthropic's Claude-3 Haiku for instance to NextChat. I have the model open come in the GUI, however, I must be doing something wrong.

Could you provide a sample docker-compose.yml or docker run command that would add a Claude-3 model?

P.S. I was running NextChat with docker compose

crbon avatar Apr 18 '24 10:04 crbon

I got it resolved.

Make sure to add of the following 3 environment variables to be able to use use Claude-3 (any model).

  • CUSTOM_MODELS
  • ANTHROPIC_API_KEY
  • ANTHROPIC_API_VERSION
  • ANTHROPIC_URL

Sample for anyone having issues, until the update arrived with better built in support for Claude-3

  • CUSTOM_MODELS="+claude-3-haiku-20240307"
  • ANTHROPIC_API_KEY="{your-api-key}"
  • ANTHROPIC_API_VERSION="2023-06-01" (currently the latest version)
  • ANTHROPIC_URL="https://api.anthropic.com/v1/messages"

crbon avatar Apr 18 '24 11:04 crbon

Sample for anyone having issues, until the update arrived with better built in support for Claude-3

* `CUSTOM_MODELS="+claude-3-haiku-20240307"`

* `ANTHROPIC_API_KEY="{your-api-key}"`

* `ANTHROPIC_API_VERSION="2023-06-01"` (currently the latest version)

* `ANTHROPIC_URL="https://api.anthropic.com/v1/messages"`

If you want to use selected models, add CUSTOM_MODELS=-all,+claude-3-sonnet-20240229,+[other models you need] in docker-compose.yml or all default models will be included in settings. Find exact model name in constant.ts.

For Claude, ANTHROPIC_API_KEY is necessary (if you use official service), others like ANTHROPIC_API_VERSION and ANTHROPIC_URL are optional. Use ANTHROPIC_URL if you access through a endpoint proxy (like third-party service provider).

Kosette avatar Apr 18 '24 12:04 Kosette

Weird. It didn't work for me till I added the ANTHROPIC_API_VERSION.

I already had the ANTHROPIC_API_KEY, ANTHROPIC_URL and CUSTOM_MODELS, but without ANTHROPIC_API_VERSION it would fallback to GPT-3

crbon avatar Apr 18 '24 12:04 crbon

This bug has beed fixed in v2.12.2

weifeng12x avatar May 09 '24 05:05 weifeng12x