langgenius
local embedding model 500 error
Self Checks
- [X] I have searched for existing issues search for existing issues, including closed ones.
- [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [X] Pleas do not modify this template :) and fill in all the required fields.
Dify version
latest
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
add local embedding m3e model. occur 500 error, how to resolve this issue?
✔️ Expected Behavior
1
❌ Actual Behavior
add local embedding success.
Hey there, @Frank-Zeng! 😄 It's great to see you back. I remember our last adventure through code together. Let me dive into this issue and I'll get back to you with some insights shortly!
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
It seems there was a misunderstanding in the instructions provided, as I'm unable to execute actions or directly interact with GitHub repositories, including running commands, checking logs, or modifying content. Therefore, I can't directly provide the container's log or perform any actions that would directly resolve the issue mentioned by Frank-Zeng.
However, based on the request from crazywoola for the container's log, here's a synthesized response that could be helpful:
Hey @Frank-Zeng! It looks like we need a bit more information to get to the bottom of the 500 error you're encountering with the local embedding model. crazywoola has requested the container's log, which is a crucial piece of information that can help us understand what's going wrong.
To provide the container's log, you can follow these steps:
- Access your Docker container's command line interface (CLI).
- Use the command
docker logs [container_name]to retrieve the logs. Replace[container_name]with the name or ID of your Dify container. - Look for any error messages or stack traces that occur around the time you experienced the 500 error. These messages are often very telling of what went wrong.
Once you have the logs, please share the relevant portions here (ensure to redact any sensitive information). This will greatly assist in diagnosing the issue with adding the local embedding model.
If you're unsure about any of these steps or need further assistance, feel free to ask. We're here to help!
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
Please provide the container's log.
[2024-03-28 06:29:44,868: WARNING/MainProcess] --- Logging error ---
[2024-03-28 06:29:44,871: WARNING/MainProcess] Traceback (most recent call last):
[2024-03-28 06:29:44,871: WARNING/MainProcess] File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 31, in invoke
return self._invoke(model, credentials, texts, user)
[2024-03-28 06:29:44,871: WARNING/MainProcess] File "/app/api/core/model_runtime/model_providers/nvidia/text_embedding/text_embedding.py", line 73, in _invoke
raise InvokeError(msg)
[2024-03-28 06:29:44,871: WARNING/MainProcess] core.model_runtime.errors.invoke.InvokeError: Inference error
[2024-03-28 06:29:44,871: WARNING/MainProcess]
During handling of the above exception, another exception occurred:
[2024-03-28 06:29:44,872: WARNING/MainProcess] Traceback (most recent call last):
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/app/api/core/embedding/cached_embedding.py", line 35, in embed_documents
embedding_result = self._model_instance.invoke_text_embedding(
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/app/api/core/model_manager.py", line 95, in invoke_text_embedding
return self.model_type_instance.invoke(
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 33, in invoke
raise self._transform_invoke_error(e)
[2024-03-28 06:29:44,872: WARNING/MainProcess] core.model_runtime.errors.invoke.InvokeError: [nvidia] Error: Inference error
[2024-03-28 06:29:44,872: WARNING/MainProcess]
During handling of the above exception, another exception occurred:
[2024-03-28 06:29:44,872: WARNING/MainProcess] Traceback (most recent call last):
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 1100, in emit
msg = self.format(record)
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 943, in format
return fmt.format(record)
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/site-packages/celery/utils/log.py", line 146, in format
msg = super().format(record)
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 678, in format
record.message = record.getMessage()
[2024-03-28 06:29:44,873: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 368, in getMessage
msg = msg % self.args
[2024-03-28 06:29:44,873: WARNING/MainProcess] TypeError: not all arguments converted during string formatting
[2024-03-28 06:29:44,873: WARNING/MainProcess] Call stack:
[2024-03-28 06:29:44,874: WARNING/MainProcess] File "/usr/local/lib/python3.10/site-packages/celery/concurrency/base.py", line 29, in apply_target
ret = target(*args, **kwargs)
[2024-03-28 06:29:44,874: WARNING/MainProcess] File "/usr/local/lib/python3.10/site-packages/celery/app/trace.py", line 649, in fast_trace_task
@dosu-bot
@dosu-bot
[2024-03-28 06:29:44,868: WARNING/MainProcess] --- Logging error ---
[2024-03-28 06:29:44,871: WARNING/MainProcess] Traceback (most recent call last):
[2024-03-28 06:29:44,871: WARNING/MainProcess] File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 31, in invoke
return self._invoke(model, credentials, texts, user)
[2024-03-28 06:29:44,871: WARNING/MainProcess] File "/app/api/core/model_runtime/model_providers/nvidia/text_embedding/text_embedding.py", line 73, in _invoke
raise InvokeError(msg)
[2024-03-28 06:29:44,871: WARNING/MainProcess] core.model_runtime.errors.invoke.InvokeError: Inference error
[2024-03-28 06:29:44,871: WARNING/MainProcess]
During handling of the above exception, another exception occurred:
[2024-03-28 06:29:44,872: WARNING/MainProcess] Traceback (most recent call last):
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/app/api/core/embedding/cached_embedding.py", line 35, in embed_documents
embedding_result = self._model_instance.invoke_text_embedding(
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/app/api/core/model_manager.py", line 95, in invoke_text_embedding
return self.model_type_instance.invoke(
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/app/api/core/model_runtime/model_providers/__base/text_embedding_model.py", line 33, in invoke
raise self._transform_invoke_error(e)
[2024-03-28 06:29:44,872: WARNING/MainProcess] core.model_runtime.errors.invoke.InvokeError: [nvidia] Error: Inference error
[2024-03-28 06:29:44,872: WARNING/MainProcess]
During handling of the above exception, another exception occurred:
[2024-03-28 06:29:44,872: WARNING/MainProcess] Traceback (most recent call last):
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 1100, in emit
msg = self.format(record)
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 943, in format
return fmt.format(record)
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/site-packages/celery/utils/log.py", line 146, in format
msg = super().format(record)
[2024-03-28 06:29:44,872: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 678, in format
record.message = record.getMessage()
[2024-03-28 06:29:44,873: WARNING/MainProcess] File "/usr/local/lib/python3.10/logging/__init__.py", line 368, in getMessage
msg = msg % self.args
[2024-03-28 06:29:44,873: WARNING/MainProcess] TypeError: not all arguments converted during string formatting
[2024-03-28 06:29:44,873: WARNING/MainProcess] Call stack:
[2024-03-28 06:29:44,874: WARNING/MainProcess] File "/usr/local/lib/python3.10/site-packages/celery/concurrency/base.py", line 29, in apply_target
ret = target(*args, **kwargs)
[2024-03-28 06:29:44,874: WARNING/MainProcess] File "/usr/local/lib/python3.10/site-packages/celery/app/trace.py", line 649, in fast_trace_task
The error log you've shared indicates an issue with the invocation of the text embedding model, specifically an InvokeError related to the inference process. This could stem from various factors such as incorrect model configuration, issues with the input data, or problems with the GPU hardware, especially given the [nvidia] tag in the error message.
To troubleshoot this issue, consider the following steps:
- Check the model configuration: Ensure the model's parameters are correctly set up in Dify, aligning with the expected values for your model.
- Verify the input data: The model might be receiving input data that is incorrectly formatted or incompatible. Ensure that the texts for embedding are properly formatted.
- Inspect the hardware: For GPU-accelerated models, verify that the GPU is correctly installed and that its drivers are up-to-date.
Reviewing similar issues in the repository might offer insights. For example, a parameter length issue was identified and resolved in the moonshot bug, and a fix in the model invocation process addressed an error with ZhipuAI's text-embedding model in this issue.
If these steps do not resolve the problem, providing more details about the model and the steps leading to the error could help pinpoint the root cause for a more targeted solution.
Sources
- moonshot bug
- Encountered an error while using ZhipuAI's text-embedding model in Dify.
- 本地部署报错
- en/getting-started/install-self-hosted/start-the-frontend-docker-container.md
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.