ragflow [Question]: The file can not be parsed, has been stuck in parsing, even for a day or two without any change。

Describe your problem

I select the corresponding knowledge base on the web page, upload multiple PDF files, start one or more file parsing, often appear stuck in the parsing (one or two days do not change), open the background log, loop the following error and warning information, please help locate the specific problem, thank you!

Jun 07 '24 03:06 Romanzhang2024

Loops of warnings and error messages： [WARNING] [2024-06-07 03:15:36,354] [synonym.init] [line:24]: Realtime synonym is disabled, since no redis connection. [WARNING] Load term.freq FAIL! Traceback (most recent call last): File "/ragflow/rag/svr/task_executor.py", line 375, in main() File "/ragflow/rag/svr/task_executor.py", line 294, in main rows = collect() File "/ragflow/rag/svr/task_executor.py", line 117, in collect assert tasks, "{} empty task!".format(msg["id"]) AssertionError: b89fcce8247811ef99fa0242ac120003 empty task! [WARNING] [2024-06-07 03:15:56,104] [synonym.init] [line:24]: Realtime synonym is disabled, since no redis connection. [INFO] [2024-06-07 03:15:56,703] [_internal._log] [line:96]: 172.18.0.6 - - [07/Jun/2024 03:15:56] "GET /v1/document/list?kb_id=6529d8601f2311efa1140242ac120006&page=1&page_size=10 HTTP/1.1" 200 - [WARNING] Load term.freq FAIL! Traceback (most recent call last): File "/ragflow/rag/svr/task_executor.py", line 375, in main() File "/ragflow/rag/svr/task_executor.py", line 294, in main rows = collect() File "/ragflow/rag/svr/task_executor.py", line 117, in collect assert tasks, "{} empty task!".format(msg["id"]) AssertionError: b89fcd4c247811ef99fa0242ac120003 empty task! [WARNING] [2024-06-07 03:16:16,005] [synonym.init] [line:24]: Realtime synonym is disabled, since no redis connection.

Jun 07 '24 03:06 Romanzhang2024

Loops of warnings and error messages： [WARNING] [2024-06-07 03:15:36,354] [synonym.init] [line:24]: Realtime synonym is disabled, since no redis connection. [WARNING] Load term.freq FAIL! Traceback (most recent call last): File "/ragflow/rag/svr/task_executor.py", line 375, in main() File "/ragflow/rag/svr/task_executor.py", line 294, in main rows = collect() File "/ragflow/rag/svr/task_executor.py", line 117, in collect assert tasks, "{} empty task!".format(msg["id"]) AssertionError: b89fcce8247811ef99fa0242ac120003 empty task! [WARNING] [2024-06-07 03:15:56,104] [synonym.init] [line:24]: Realtime synonym is disabled, since no redis connection. [INFO] [2024-06-07 03:15:56,703] [_internal._log] [line:96]: 172.18.0.6 - - [07/Jun/2024 03:15:56] "GET /v1/document/list?kb_id=6529d8601f2311efa1140242ac120006&page=1&page_size=10 HTTP/1.1" 200 - [WARNING] Load term.freq FAIL! Traceback (most recent call last): File "/ragflow/rag/svr/task_executor.py", line 375, in main() File "/ragflow/rag/svr/task_executor.py", line 294, in main rows = collect() File "/ragflow/rag/svr/task_executor.py", line 117, in collect assert tasks, "{} empty task!".format(msg["id"]) AssertionError: b89fcd4c247811ef99fa0242ac120003 empty task! [WARNING] [2024-06-07 03:16:16,005] [synonym.init] [line:24]: Realtime synonym is disabled, since no redis connection.

This happens when the data is dirty after restarting ragflow. Ignore them, and cancel the task on the web page.

Jun 07 '24 07:06 KevinHuSh

Describe your problem

I select the corresponding knowledge base on the web page, upload multiple PDF files, start one or more file parsing, often appear stuck in the parsing (one or two days do not change), open the background log, loop the following error and warning information, please help locate the specific problem, thank you!

If the progressing bar has not changed for 5 seconds at the begining, there must be some thing wrong with system. You could cancle and redo. Typically, the embedding and indexing procedures are very slow if only CPUs are used.

Jun 07 '24 07:06 KevinHuSh

1、The configuration file docker-compose.yml in the docker directory has specified that two GPU resources A10 are used deploy: resources: reservations: devices: - driver: nvidia device_ids: ['0','1']
capabilities: [gpu] 2、Look at the web page parsing progress bar (progress 40.91%, experienced about 5 hours，Long time no change) prompt message, found the following Loop error： Page(85~97): [ERROR]Embedding error:Account abnormal. Please ensure it's on good standing. Page(97~109): [ERROR]Embedding error:Account abnormal. Please ensure it's on good standing.

Jun 07 '24 08:06 Romanzhang2024

1、The configuration file docker-compose.yml in the docker directory has specified that two GPU resources A10 are used deploy: resources: reservations: devices: - driver: nvidia device_ids: ['0','1'] capabilities: [gpu] 2、Look at the web page parsing progress bar (progress 40.91%, experienced about 5 hours，Long time no change) prompt message, found the following Loop error： Page(85~97): [ERROR]Embedding error:Account abnormal. Please ensure it's on good standing. Page(97~109): [ERROR]Embedding error:Account abnormal. Please ensure it's on good standing.

Please check the status of embedding you've choosen. Is it based on SaaS. If it is, you need to set API key in system settings.

Jun 07 '24 11:06 KevinHuSh

I have configured The LLM factory（service_conf.yaml：user_default_llm, ali:Tongyi-Qianwen ） and the corresponding api_key.

Jun 08 '24 09:06 Romanzhang2024

Have you solved this yet? I've encountered the same problem too.