dify icon indicating copy to clipboard operation
dify copied to clipboard

Inconsistency across deployment mediums regarding Anthropic's Sonnet 3.5 model.

Open sid-newby opened this issue 8 months ago • 1 comments

Self Checks

  • [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
  • [X] I have searched for existing issues search for existing issues, including closed ones.
  • [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.11

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

The new Sonnet model ( claude-3-5-sonnet-20240620 ) is successfully returned as an option when configuring the provider (Anthropic API) after entering an API key in the cloud edition.

After performing the same operation with the docker deployment, all models except claude-3-5-sonnet-20240620 return. Google Chrome console reports 404, the web app throws an error modal at the top of the page.

Source code included with git repo contains /app/api/core/model_runtime/model_providers/anthropic/llm/claude-3-5-sonnet-20240620.yaml and _position.yaml contains definition for claude-3-5-sonnet-20240620.

Docker image does not contain the model's .yaml and the definition is missing from the position.yaml.

I was able to resolve this by simply modifying and saving the _position.yaml file and uploading my git source provided claude-3-5-sonnet-20240620.yaml to the llm directory under /anthropic.

In the same vein, I noticed the docker distribution has a number of vulnerabilities that i'd like to patch out. Do you guys maintain the image in that respect, or should I wait for updates and pull?

Thanks.

✔️ Expected Behavior

Consistency across web hosted, containerized and distributed source code and functionality.

❌ Actual Behavior

Troubleshooting and inconsistency across the mediums, leading to 3 hours of my life gone. 1  Localhost-Docker OpenAI Compatible Config Response 2  Localhost-Docker Anthropic Successful Configuration 3  Localhost-Docker Anthropic Fails to Return claude-3-5-sonnet-20240620 4  Cloud Hosted Model Anthropic Successfully Returns claude-3-5-sonnet-20240620 5  GitHub Source Code_ Model Definition _position yaml for Anthropic Configured for claude-3-5-sonnet-20240620 6  Docker Source Code_ Model Definition Anthropic Not Configured for claude-3-5-sonnet-20240620 7  Localhost-Docker Anthropic Now Returns claude-3-5-sonnet-20240620

sid-newby avatar Jun 24 '24 10:06 sid-newby