AWS sagemaker LLM support

[benjamin-code]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to file this report (我已阅读并同意 Language Policy).

1. Is this request related to a challenge you're experiencing?

We have some open source llm and embedded model running in AWS sagemaker.How can I access these models in dify?

2. Describe the feature you'd like to see

Access models running in AWS sagemaker

3. How will this feature improve your workflow or experience?

Sagemaker is enterprise LLM platform in our global company

4. Additional context or comments

No response

5. Can you help us with this feature?

• [X] I am interested in contributing to this feature.

[dosubot[bot]]

🤖

Hello again, @benjamin-code! It's great to have you back contributing your ideas. Hope you're doing well!

Based on the information available in the dify repository, it appears that dify does have the capability to integrate with external LLM platforms. This is evidenced by the existing integrations with OpenAI's GPT-3.5, GPT-4, and text-davinci-003 models, as seen in the api/core/model_runtime/model_providers/openai/llm/_position.yaml file.

However, there is no explicit mention or evidence of support for AWS Sagemaker in the current version (v0.4.8) of dify. This doesn't necessarily mean that it's not possible, but it may require additional work or customization on your part.

Since you've expressed interest in contributing to this feature, I would recommend starting by examining the existing integrations with OpenAI in the aforementioned YAML file. This could provide a useful starting point for understanding how dify interacts with external LLM platforms.

If you need further assistance or have more specific questions about how to implement this, please provide more details and I'll do my best to help.

Sources

• api/core/model_runtime/model_providers/openai/llm/_position.yaml

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[takatost]

Thx a lot for your interest in contributing to Dify! 😁👍 You can check out the following docs on how to integrate a new model provider: https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md https://github.com/langgenius/dify/blob/main/api/core/model_runtime/README.md

[canny[bot]]

This issue has been linked to a Canny post: AWS Sagemaker :tada:

[github-actions[bot]]

Close due to it's no longer active, if you have any questions, you can reopen it.