Replicate API mismatch

[deepfates]

Self Checks

• [X] This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Pleas do not modify this template :) and fill in all the required fields.

Dify version

0.6.4

Cloud or Self Hosted

Cloud

Steps to reproduce

Attempt to set meta/meta-llama-3-70b-instruct model through Replicate

✔️ Expected Behavior

Language models with token-based pricing don't have versions on Replicate anymore. I should be able to set [no version] somehow so that the form can validate.

❌ Actual Behavior

The form doesn't want to let me click Send and when i enter an empty string in the version input it won't validate

[dosubot[bot]]

It seems like you're encountering an issue where the Dify platform requires a model_version for Replicate models, even when you're dealing with models that don't have versions. The validation step in the code checks for the presence of a model_version and will not proceed without it, leading to the problem you're facing.

A practical workaround for now is to use a placeholder value in the model_version field, such as "latest", "default", or "n/a". This can help bypass the validation requirement and allow you to proceed with your setup. Keep in mind, this is a temporary solution, and for a more permanent fix, it would be best to reach out to Replicate's support or consult their official documentation for guidance on handling models without versions.

Details

Sources

• api/core/model_runtime/model_providers/replicate/replicate.yaml
• api/core/model_runtime/model_providers/replicate/llm/llm.py
• api/core/model_runtime/model_providers/replicate/text_embedding/text_embedding.py

About Dosu This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[deepfates]

Thank you @dosu robot. I did try this but it gives error Model meta/meta-llama-3-70b-instruct:latest not exists, cause: ReplicateError:Not found.

[dosubot[bot]]

It appears the issue stems from the Dify platform's requirement for a model version when integrating Replicate models, which conflicts with models that do not traditionally have versions, such as the one you're trying to use. The error you encountered when using a placeholder like "latest" indicates that the platform is specifically looking for a valid version that matches its records, and placeholders are not recognized as valid versions.

Given the constraints of the Dify platform and its current integration capabilities with Replicate, there isn't a straightforward workaround for integrating models that don't have explicit versions. The platform's validation process is designed to ensure that a model version is specified, as seen in the error message you received, which suggests that the platform's backend is checking for a specific, supported version of the model.

Unfortunately, without a direct way to bypass this requirement or specify a non-version for such models, the best course of action would be to reach out to the Dify platform's support team or consult the Replicate documentation for any updates or changes in how they handle model versions. This might provide a more definitive solution or an update that allows for the integration of models without versions.

In the meantime, ensuring that all required fields are correctly filled with valid information according to Replicate's and Dify's current specifications is essential. If there are any updates or changes in the platform's handling of model versions, those would likely be your best bet for successfully integrating the model you're working with.

Details

Sources

• en/tutorials/model-configuration/replicate.md
• api/core/model_runtime/model_providers/replicate/llm/llm.py
• api/core/model_runtime/model_providers/replicate/replicate.yaml
• en/tutorials/model-configuration/README.md

About Dosu This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.