infiniflow
[Bug]: Task_executor.py high cpu usage (idle)
Self Checks
- [x] I have searched for existing issues search for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report (Language Policy).
- [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
- [x] Please do not modify this template :) and fill in all the required fields.
RAGFlow workspace code commit ID
v0.17.2-slim
RAGFlow image version
v0.17.2-slim
Other environment information
Hardware: Dell Optiplex 9020 Mini Tower Desktop PC, Intel Core i7-4770-3.4 GHz, 32GB DDR3 Ram, NVIDIA GeForce GTX 1050 Ti
Environment: Proxmox LXC (Privileged) | Ubuntu 22.04.5 LTS
Docker version: 28.0.4
Docker compose version: v2.34.0
Actual behavior
Application runs without issue except for high cpu usage when idle. "htop" shows an average 96% cpu usage for "python3 rag/svr/task_executor.py 0" when idle.
System page
Docker ps
9e577383f501 infiniflow/ragflow:v0.17.2-slim "./entrypoint.sh" 19 minutes ago Up 19 minutes 0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp, 0.0.0.0:9380->9380/tcp, [::]:9380->9380/tcp ragflow-server
f5b8fc17bf57 elasticsearch:8.11.3 "/bin/tini -- /usr/l…" 19 minutes ago Up 19 minutes (healthy) 9300/tcp, 0.0.0.0:1200->9200/tcp, [::]:1200->9200/tcp ragflow-es-01
332b38452c70 quay.io/minio/minio:RELEASE.2023-12-20T01-00-02Z "/usr/bin/docker-ent…" 19 minutes ago Up 19 minutes 0.0.0.0:9000-9001->9000-9001/tcp, [::]:9000-9001->9000-9001/tcp ragflow-minio
a3ed0f80d4d5 mysql:8.0.39 "docker-entrypoint.s…" 19 minutes ago Up 19 minutes (healthy) 33060/tcp, 0.0.0.0:5455->3306/tcp, [::]:5455->3306/tcp ragflow-mysql
66cf3b635e4d valkey/valkey:8 "docker-entrypoint.s…" 19 minutes ago Up 19 minutes 0.0.0.0:6379->6379/tcp, [::]:6379->6379/tcp ragflow-redis
Docker Logs for ragflow-server
2025-04-03 22:33:57,590 WARNING 21 RedisDB.queue_info rag_flow_svr_queue got exception: no such key 2025-04-03 22:33:57,590 INFO 21 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2025-04-03T22:33:57.590+08:00", "boot_at": "2025-04-03T22:13:25.925+08:00", "pending": 0, "lag": 0, "done": 0, "failed": 0, "current": {}}
Docker Logs for ragflow-mysql
2025-04-03 22:12:40+08:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.39-1.el9 started. 2025-04-03 22:12:40+08:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' 2025-04-03 22:12:40+08:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.39-1.el9 started. '/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock' 2025-04-03T14:12:41.323048Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead. 2025-04-03T14:12:41.324056Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead. 2025-04-03T14:12:41.324076Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.39) starting as process 1 2025-04-03T14:12:41.441148Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2025-04-03T14:12:42.384635Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. 2025-04-03T14:12:42.737507Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed. 2025-04-03T14:12:42.737540Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel. 2025-04-03T14:12:42.743404Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory. 2025-04-03T14:12:42.835263Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock 2025-04-03T14:12:42.835325Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.39' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL. 2025-04-03T14:12:50.070077Z 9 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
Docker Logs for ragflow-redis
WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. 1:M 03 Apr 2025 14:12:40.709 * oO0OoO0OoO0Oo Valkey is starting oO0OoO0OoO0Oo 1:M 03 Apr 2025 14:12:40.709 * Valkey version=8.1.0, bits=64, commit=00000000, modified=0, pid=1, just started 1:M 03 Apr 2025 14:12:40.709 * Configuration loaded 1:M 03 Apr 2025 14:12:40.709 * monotonic clock: POSIX clock_gettime 1:M 03 Apr 2025 14:12:40.710 * Running mode=standalone, port=6379. 1:M 03 Apr 2025 14:12:40.710 * Server initialized 1:M 03 Apr 2025 14:12:40.710 * Loading RDB produced by Valkey version 8.1.0 1:M 03 Apr 2025 14:12:40.710 * RDB age 458 seconds 1:M 03 Apr 2025 14:12:40.710 * RDB memory usage when created 1.24 Mb 1:M 03 Apr 2025 14:12:40.710 * Done loading RDB, keys loaded: 2, keys expired: 1. 1:M 03 Apr 2025 14:12:40.710 * DB loaded from disk: 0.000 seconds 1:M 03 Apr 2025 14:12:40.710 * Ready to accept connections tcp 1:M 03 Apr 2025 14:17:41.040 * 100 changes in 300 seconds. Saving... 1:M 03 Apr 2025 14:17:41.040 * Background saving started by pid 14 14:C 03 Apr 2025 14:17:41.054 * DB saved on disk 14:C 03 Apr 2025 14:17:41.054 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 1:M 03 Apr 2025 14:17:41.141 * Background saving terminated with success 1:M 03 Apr 2025 14:22:42.015 * 100 changes in 300 seconds. Saving... 1:M 03 Apr 2025 14:22:42.016 * Background saving started by pid 15 15:C 03 Apr 2025 14:22:42.030 * DB saved on disk 15:C 03 Apr 2025 14:22:42.030 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 1:M 03 Apr 2025 14:22:42.116 * Background saving terminated with success 1:M 03 Apr 2025 14:27:43.070 * 100 changes in 300 seconds. Saving... 1:M 03 Apr 2025 14:27:43.071 * Background saving started by pid 16 16:C 03 Apr 2025 14:27:43.086 * DB saved on disk 16:C 03 Apr 2025 14:27:43.086 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 1:M 03 Apr 2025 14:27:43.171 * Background saving terminated with success 1:M 03 Apr 2025 14:32:44.014 * 100 changes in 300 seconds. Saving... 1:M 03 Apr 2025 14:32:44.014 * Background saving started by pid 17 17:C 03 Apr 2025 14:32:44.028 * DB saved on disk 17:C 03 Apr 2025 14:32:44.028 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 1:M 03 Apr 2025 14:32:44.115 * Background saving terminated with success 1:M 03 Apr 2025 14:37:45.072 * 100 changes in 300 seconds. Saving... 1:M 03 Apr 2025 14:37:45.073 * Background saving started by pid 18 18:C 03 Apr 2025 14:37:45.092 * DB saved on disk 18:C 03 Apr 2025 14:37:45.092 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 1:M 03 Apr 2025 14:37:45.173 * Background saving terminated with success
Expected behavior
No response
Steps to reproduce
$ sysctl -w vm.max_map_count=262144
$ git clone https://github.com/infiniflow/ragflow.git
$ cd ragflow/docker
$ git checkout -f v0.17.2
$ docker compose -f docker-compose-gpu.yml up -d
Additional information
I have not modified any docker compose yaml file nor the service_conf.yaml. I'm aware that Ragflow generally requires more resources but the high cpu usage when idle seems off.
Hey there, @BiigBeaker! I'm here to help you with any bugs, questions, or contributions you might have regarding RAGFlow. Let's squash those bugs together!
The high CPU usage in "task_executor.py" when idle is likely due to the frequent execution of the report_status function. This function performs operations such as deep copying of CURRENT_TASKS, JSON serialization, and Redis operations every 30 seconds. These operations can be CPU-intensive, especially if CURRENT_TASKS is large or complex. The function runs continuously in a loop with a 30-second delay, which can cause high CPU usage even when the system is idle [1].
To mitigate this, you can adjust the frequency of the report_status function by changing the await trio.sleep(30) statement to a longer interval, such as await trio.sleep(60), to reduce the frequency of these operations [1].
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
After try all way to fix this. I finally know how to fix this. you trust login to your site. upload a knowledge base, and create an assistant , and try to make some chat
After try all way to fix this. I finally know how to fix this. you trust login to your site. upload a knowledge base, and create an assistant , and try to make some chat
This is the answer. Immediately after parsing the files that I uploaded to a new knowledge base, my system usage dropped drastically. I checked the usage for task_executor afterwards and now it averages around 1% cpu usage!
I checked docker logs for ragflow-server as well and noticed RedisDB.queue_info rag_flow_svr_queue got exception: no such key was no longer appearing. My guess is that the redis key that the task_executor was expecting is not created until after parsing your first file(s) uploaded to a knowledge base. So, until parsing your first file, the task_executor constantly polls redis for a key not yet created which causes the high usage
After try all way to fix this. I finally know how to fix this. you trust login to your site. upload a knowledge base, and create an assistant , and try to make some chat
This is indeed a solution
