langgenius
PluginDaemonInternalServerError - Vertex AI API Connection Timeout
Self Checks
- [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
- [x] I have searched for existing issues search for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
- [x] Please do not modify this template :) and fill in all the required fields.
Dify version
1.0.0
Cloud or Self Hosted
- Self Hosted (Docker)
- Cloud
Steps to reproduce
- Navigate to workspace settings
- Select model providers configuration
- Choose "Vertex AI" provider
- Enter credentials and submit the form
- Wait for validation process
✔️ Expected Behavior
Vertex AI credentials are stored and Vertex models are usable
❌ Actual Behavior
The save button is disabled for a couple minutes, and re-enables without any errors.
Docker logs:
2025-02-28 21:32:30.216 ERROR [Dummy-38] [app.py:875] - Exception on /console/api/workspaces/current/model-providers/langgenius/vertex_ai/vertex_ai [POST]
Traceback (most recent call last):
File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 917, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask/app.py", line 902, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/__init__.py", line 489, in wrapper
resp = resource(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask/views.py", line 110, in view
return current_app.ensure_sync(self.dispatch_request)(**kwargs) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/.venv/lib/python3.12/site-packages/flask_restful/__init__.py", line 604, in dispatch_request
resp = meth(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/wraps.py", line 147, in decorated
return view(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/libs/login.py", line 94, in decorated_view
return current_app.ensure_sync(func)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/wraps.py", line 27, in decorated
return view(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/api/controllers/console/workspace/model_providers.py", line 102, in post
model_provider_service.save_provider_credentials(
File "/app/api/services/model_provider_service.py", line 145, in save_provider_credentials
provider_configuration.add_or_update_custom_credentials(credentials)
File "/app/api/core/entities/provider_configuration.py", line 248, in add_or_update_custom_credentials
provider_record, credentials = self.custom_credentials_validate(credentials)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/entities/provider_configuration.py", line 231, in custom_credentials_validate
credentials = model_provider_factory.provider_credentials_validate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 155, in provider_credentials_validate
self.plugin_model_manager.validate_provider_credentials(
File "/app/api/core/plugin/manager/model.py", line 97, in validate_provider_credentials
for resp in response:
^^^^^^^^
File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
self._handle_plugin_daemon_error(error.error_type, error.message)
File "/app/api/core/plugin/manager/base.py", line 223, in _handle_plugin_daemon_error
raise PluginDaemonInternalServerError(description=message)
core.plugin.manager.exc.PluginDaemonInternalServerError: PluginDaemonInternalServerError: killed by timeout
Hey, @jerry-udensi! I'm here to help you with any bugs, questions, or contributions you have for the Dify project. Let's work together to solve this issue!
I found a similar unsolved discussion titled "What is the reason for the request timed out when saving the model?" which might be related to the timeout issue you're experiencing [1].
The PluginDaemonInternalServerError exception is typically used in the Dify codebase to signal internal server errors within the plugin daemon, indicating a server-side issue rather than a client-side one [2].
For handling API connection timeouts, the Dify platform implements a retry mechanism. This includes a "Retry on Failure" feature that automatically retries requests according to a preset strategy. You can adjust the maximum number of retries and the interval between each retry. The retry mechanism is triggered for specific HTTP status codes such as 429, 500, 502, 503, and 504 [3][4].
Given the error message "killed by timeout," it seems that the connection to the Vertex AI API is timing out. You might want to check your network connection, ensure that the Vertex AI API is accessible, and consider increasing the timeout settings if possible. Additionally, reviewing the retry settings might help mitigate the issue.
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
Given the error message "killed by timeout," it seems that the connection to the Vertex AI API is timing out. You might want to check your network connection, ensure that the Vertex AI API is accessible, and consider increasing the timeout settings if possible. Additionally, reviewing the retry settings might help mitigate the issue.
All other models I use (Claude, Azure OpenAI) work just fine. Its worth mentioning I upgraded from 0.5.3 to 1.0.0 @dosu
To address the timeout issue with Vertex AI after upgrading from Dify version 0.5.3 to 1.0.0, you can try adjusting the timeout settings in the .env file. Here’s how you can do it:
-
If you haven't already, create a
.envfile by copying from.env.example:cp .env.example .env -
In the
.envfile, configure the following environment variables to increase the timeout values:SSRF_DEFAULT_TIME_OUTSSRF_DEFAULT_CONNECT_TIME_OUTSSRF_DEFAULT_READ_TIME_OUTSSRF_DEFAULT_WRITE_TIME_OUTHTTP_REQUEST_MAX_CONNECT_TIMEOUTHTTP_REQUEST_MAX_READ_TIMEOUTHTTP_REQUEST_MAX_WRITE_TIMEOUT
For example, you can set
HTTP_REQUEST_MAX_READ_TIMEOUT=600andHTTP_REQUEST_MAX_WRITE_TIMEOUT=600to allow for longer read and write operations [1].
Since other models like Claude and Azure OpenAI are working fine, the issue might be specific to the Vertex AI configuration or the changes introduced in the upgrade. Adjusting these settings should help mitigate the timeout problem.
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
To address the timeout issue with Vertex AI after upgrading from Dify version 0.5.3 to 1.0.0, you can try adjusting the timeout settings in the
.envfile. Here’s how you can do it:
If you haven't already, create a
.envfile by copying from.env.example: cp .env.example .envIn the
.envfile, configure the following environment variables to increase the timeout values:
SSRF_DEFAULT_TIME_OUTSSRF_DEFAULT_CONNECT_TIME_OUTSSRF_DEFAULT_READ_TIME_OUTSSRF_DEFAULT_WRITE_TIME_OUTHTTP_REQUEST_MAX_CONNECT_TIMEOUTHTTP_REQUEST_MAX_READ_TIMEOUTHTTP_REQUEST_MAX_WRITE_TIMEOUTFor example, you can set
HTTP_REQUEST_MAX_READ_TIMEOUT=600andHTTP_REQUEST_MAX_WRITE_TIMEOUT=600to allow for longer read and write operations [1].Since other models like Claude and Azure OpenAI are working fine, the issue might be specific to the Vertex AI configuration or the changes introduced in the upgrade. Adjusting these settings should help mitigate the timeout problem.
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
That didnt help.
Please try https://github.com/langgenius/dify-official-plugins/pull/342
