rag workflow and hiting-test timeout

[yuanzhiwei]

Self Checks

• [x] I have read the Contributing Guide and Language Policy.
• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report, otherwise it will be closed.
• [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.1

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

1、Run hiting-test or RAG Retrieval After waiting for a long time, the following message appeared.

Did the Worker restart?

The same problem applies if there is knowledge retrieval in the workflow.

✔️ Expected Behavior

Run normal

❌ Actual Behavior

Knowledge retrieval will time out, and the backend API log will output the content shown in the image below.

[dosubot[bot]]

Your RAG workflow and hiting-test timeouts, along with Gunicorn worker restarts and gevent monkey-patching warnings, are most likely caused by a combination of (1) long-running or resource-intensive retrieval steps in your workflow, (2) conservative default Gunicorn/gevent timeout and concurrency settings, and (3) potential issues with the order of gevent monkey-patching in your deployment.

Here’s what you can do:

• Increase timeouts: Set higher values for GUNICORN_TIMEOUT, WORKFLOW_MAX_EXECUTION_TIME, PLUGIN_MAX_EXECUTION_TIMEOUT, and related environment variables in your .env or docker-compose.yaml (e.g., 600 or 1800 seconds). Make sure NGINX proxy timeouts are also set higher than your workflow timeouts, or you’ll get 504 errors from the proxy instead of the app [details].
• Tune concurrency: Increase SERVER_WORKER_AMOUNT (ideally to CPU cores x 2 + 1) and SERVER_WORKER_CONNECTIONS to handle more parallel requests. Set APP_MAX_ACTIVE_REQUESTS to a reasonable value to avoid overwhelming the server. Make sure your database pool (SQLALCHEMY_POOL_SIZE) is large enough for your expected load [details].
• Check gevent monkey-patching: The warning about monkey-patching ssl after import means some modules are imported before gevent patches the standard library, which can cause recursion errors and worker instability. Dify’s api/gunicorn.conf.py tries to patch gRPC and psycopg2 after gevent’s patching event, but if you import anything that uses SSL (like requests) too early, you’ll hit this bug. Avoid top-level imports in Gunicorn config and ensure all monkey-patching happens as early as possible [details].
• Consider switching worker class: If you see persistent threading or timeout errors with SERVER_WORKER_CLASS=gevent, try switching to uvicorn as the worker class to avoid gevent/threading conflicts [details].
• Monitor resource usage: High CPU or memory usage during retrieval can cause workers to be killed. Monitor your server and consider scaling up if needed [details].
• Apply and restart: After changing environment variables, rebuild and restart all containers to apply the new settings.

Timeouts in RAG/knowledge retrieval are often due to slow metadata filtering, reranking, or large datasets. If you have custom plugins or heavy retrieval logic, check their logs and consider optimizing them. If you’re running a version of gevent that’s incompatible with your Python version (e.g., gevent 25.x with Python 3.12), downgrade gevent to 24.11.1 [details].

If you’re still seeing timeouts after these changes, check Dify, Gunicorn, and NGINX logs to see which layer is terminating the request. Recent Dify releases include fixes for concurrency and workflow stability, so upgrading may also help.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[yuanzhiwei]

My knowledge base doesn't have many documents; there's only one test document.

[yuanzhiwei]

This is my startup script， It was working fine before, but it's unusable after the upgrade.

[dosubot[bot]]

Your script stopped working after the upgrade because Dify 1.11.1 introduced several breaking changes and new requirements for how Gunicorn, gevent, and Celery are started and patched.

Key issues:

• The order of gevent monkey-patching is now critical. If any module (like requests or anything using SSL) is imported before gevent and gRPC/psycopg2 are patched, you’ll get recursion errors and worker restarts. Dify’s official startup uses a special gunicorn.conf.py and celery_entrypoint.py to ensure patching happens at the right time. Custom scripts must mirror this order or you’ll see the “monkey-patching ssl after import” warning and instability details.
• The Celery worker queue list has changed. You must now include all required queues (like priority_dataset, workflow, schedule_poller, etc.) or tasks will not be processed and you’ll see timeouts details.
• Running both Flask and Gunicorn on the same port is not supported—use only Gunicorn for production, and only Flask for debugging.
• The gevent version was recently updated. If you’re using Python 3.12, gevent 25.x can cause recursion errors with SSL. If you see these, downgrade gevent to 24.11.1 details.

How to fix:

1. Update your Celery worker command to use the full queue list required by Dify 1.11.1.
2. Only start Gunicorn (not Flask) on port 5001 for production.
3. Make sure your script does not import any modules that use SSL or psycopg2 before Gunicorn or Celery start and apply monkey-patching. If you’re not sure, use the official entrypoint logic as a template details.
4. If you see recursion errors, check your gevent version and downgrade if needed.

If you want a robust setup, use the official entrypoint script or Docker Compose as your baseline. This will ensure all patching, queue, and worker settings are correct for the new version.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[fatelei]

it has been optimized