[Bug]: When the knowledge graph is enabled, retrieval testing throw error: JSONDecodeError('Expecting value: line 1 column 1 (char 0)')

[tevooli]

Is there an existing issue for the same bug?

• [x] I have checked the existing issues.

RAGFlow workspace code commit ID

v0.16.0

RAGFlow image version

v0.16.0 full

Other environment information

system chat model: qwen-max

Actual behavior

ragflow-server  | 2025-02-11 14:42:02,509 INFO     15 POST http://es01:9200/ragflow_1f7c4da4e82a11ef85130242ac1b0006/_search [status:200 duration:0.008s]
ragflow-server  | 2025-02-11 14:42:02,511 ERROR    15 Expecting value: line 1 column 1 (char 0)
ragflow-server  | Traceback (most recent call last):
ragflow-server  |   File "/ragflow/api/apps/chunk_app.py", line 311, in retrieval_test
ragflow-server  |     ck = settings.kg_retrievaler.retrieval(question,
ragflow-server  |   File "/ragflow/graphrag/search.py", line 251, in retrieval
ragflow-server  |     "Description": json.loads(rel["description"]).get("description", "")
ragflow-server  |   File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
ragflow-server  |     return _default_decoder.decode(s)
ragflow-server  |   File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
ragflow-server  |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
ragflow-server  |   File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
ragflow-server  |     raise JSONDecodeError("Expecting value", s, err.value) from None
ragflow-server  | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
ragflow-server  | 2025-02-11 14:42:02,513 INFO     15 172.30.0.6 - - [11/Feb/2025 14:42:02] "POST /v1/chunk/retrieval_test HTTP/1.1" 200 -

Expected behavior

No response

Steps to reproduce

1. create knowledge base, configuration:
  
  Document language: Chinese
  Embedding model: BAAI/bge-large-zh-v1.5
  Chunk method: General
  Page rank: 0
  Auto-keyword: 0
  Auto-question: 0
  Chunk token number: 512
  Delimiters: \n!?;。；！？
  Layout recognition & OCR: DeepDOC
  Excel to HTML: false
  Use RAPTOR to enhance retrieval: false
  Extract knowledge graph: true
  ENtity types: organization, person, geo, event, category
  Method: Light
  Entity resolution: true
  Community reports generation: false
  Tag set: null
  
2. upload a document
3. Retrieval testing, and enable `Use knowledge graph`

Additional information

No response

[KevinHuSh]

This is fixed. Upgrade to nightly version of docker image.

[TinyFatty]

After switching to nightly version , Still has the same problem

configuration:
  Document language: Chinese
  Embedding model: OpenAI/text-embedding-3-large
  Chunk method: General
  Page rank: 0
  Auto-keyword: 0
  Auto-question: 0
  Chunk token number: 128
  Delimiters: \n!?;。；！？
  Layout recognition & OCR: DeepDOC
  Excel to HTML: false
  Use RAPTOR to enhance retrieval: false
  Extract knowledge graph: true
  ENtity types: organization, person, geo, event, category
  Method: Light
  Entity resolution: true
  Community reports generation: false
  Tag set: null

In the webpage, it shows "SUCCESS" but with knowledge graph error like this

Begin at:
Tue, 25 Feb 2025 18:06:04 GMT
Duration:
190.24 s
Progress:
18:06:04 Task has been received.
18:06:05 Page(1~13): OCR started
18:06:12 Page(1~13): OCR finished (6.51s)
18:06:34 Page(1~13): Layout analysis (22.33s)
18:06:35 Page(1~13): Table analysis (1.06s)
18:06:35 Page(1~13): Text merged (0.00s)
18:06:36 Page(1~13): Generate 7 chunks
18:06:38 Page(1~13): Embedding chunks (1.68s)
18:06:38 Page(1~13): Done (0.16s)
18:06:38 Task has been received.
18:06:39 Page(13~19): OCR started
18:06:43 Page(13~19): OCR finished (3.54s)
18:07:07 Page(13~19): Layout analysis (24.01s)
18:07:08 Page(13~19): Table analysis (0.98s)
18:07:08 Page(13~19): Text merged (0.00s)
18:07:08 Page(13~19): Generate 2 chunks
18:07:14 Page(13~19): Embedding chunks (5.88s)
18:07:14 Page(13~19): Done (0.13s)
18:07:19 Entities extraction progress
18:07:19 Task has been received.
18:08:21 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:08:21 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge graph extraction error:Expecting value: line 1 column 1 (char 0)
18:09:08 Knowledge Graph is done (107.96s)

In the log file, it shows JSONDecodeError

2025-02-25 18:08:21,292 ERROR    24 error extracting graph
Traceback (most recent call last):
  File "/ragflow/graphrag/light/graph_extractor.py", line 95, in _process_single_content
    final_result = self._chat(hint_prompt, [{"role": "user", "content": "Output:"}], gen_conf)
  File "/ragflow/graphrag/general/extractor.py", line 62, in _chat
    response = self._llm.chat(system, hist, conf)
  File "/ragflow/api/db/services/llm_service.py", line 287, in chat
    txt, used_tokens = self.mdl.chat(system, history, gen_conf)
  File "/ragflow/rag/llm/chat_model.py", line 46, in chat
    response = self.client.chat.completions.create(
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_utils/_utils.py", line 274, in wrapper
    return func(*args, **kwargs)
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 704, in create
    return self._post(
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1260, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 937, in request
    return self._request(
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1043, in _request
    return self._process_response(
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1142, in _process_response
    return api_response.parse()
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_response.py", line 314, in parse
    parsed = self._parse(to=to)
  File "/ragflow/.venv/lib/python3.10/site-packages/openai/_response.py", line 256, in _parse
    data = response.json()
  File "/ragflow/.venv/lib/python3.10/site-packages/httpx/_models.py", line 764, in json
    return jsonlib.loads(self.content, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I tried to use different Embedding Models(gpt, deepseek online api and deepseek in local ollama),but get the same error every time

[AAlexDing]

Me,too. Any solutions here?

[joedong]

Me,too. Any solutions here?

[AAlexDing]

I tried another chat model without a rate limit, and it works. Maybe the lack of a retry method or load-balancing caused this error?

[TinyFatty]

Update: It seems the knowledge graph extraction use "Chat Model" in System Model Setting rather than the embedding model in Knowledge base. I switched "Chat Model" to gpt-4o-mini and it worked at first. Howerver it shows "rate limit exceeded" quickly.Any idea how to manually slow down the TPM? Here is the error log:

14:47:48 Task has been received.
14:47:48 Page(1~100000001): Start to parse.
14:47:49 Page(1~100000001): Finish parsing.
14:47:49 Page(1~100000001): Generate 47 chunks
14:47:54 Page(1~100000001): Embedding chunks (4.89s)
14:47:55 Page(1~100000001): Done (0.35s)
14:47:56 Start RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval).
14:47:56 Task has been received.
14:48:03 Cluster one layer: 47 -> 7
14:48:07 Cluster one layer: 7 -> 4
14:48:16 Cluster one layer: 4 -> 3
14:48:20 Cluster one layer: 3 -> 2
14:48:23 Cluster one layer: 2 -> 1
14:48:23 Done (0.16s)
14:48:54 Knowledge graph extraction error:**ERROR**: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o-mini in organization org-XXXXXXXXXXXXXXXXXXXXXXXX on tokens per min (TPM): Limit 200000, Used 199481, Requested 2126. Please try again in 482ms. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
14:48:54 Knowledge graph extraction error:**ERROR**: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o-mini in organization org-XXXXXXXXXXXXXXXXXXXXXXXX on tokens per min (TPM): Limit 200000, Used 199833, Requested 2112. Please try again in 583ms. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

[KevinHuSh]

Set the envirement variable export GRAPH_EXTRACTOR_MAX_WORKERS=2

[TinyFatty]

Set the envirement variable export GRAPH_EXTRACTOR_MAX_WORKERS=2

Thx，it worked now. Btw,what's the default value of GRAPH_EXTRACTOR_MAX_WORKERS?

[KevinHuSh]

[GuoDapeng]

commit ID : 78b2e0be890fd7b3913bbc9ca26c70e3652dc566

I have used this version to test the knowledge graph with qwen2.5:7b。I built the image test myself

[sammichenVV]

Update: It seems the knowledge graph extraction use "Chat Model" in System Model Setting rather than the embedding model in Knowledge base. I switched "Chat Model" to gpt-4o-mini and it worked at first. Howerver it shows "rate limit exceeded" quickly.Any idea how to manually slow down the TPM? Here is the error log:

14:47:48 Task has been received.
14:47:48 Page(1~100000001): Start to parse.
14:47:49 Page(1~100000001): Finish parsing.
14:47:49 Page(1~100000001): Generate 47 chunks
14:47:54 Page(1~100000001): Embedding chunks (4.89s)
14:47:55 Page(1~100000001): Done (0.35s)
14:47:56 Start RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval).
14:47:56 Task has been received.
14:48:03 Cluster one layer: 47 -> 7
14:48:07 Cluster one layer: 7 -> 4
14:48:16 Cluster one layer: 4 -> 3
14:48:20 Cluster one layer: 3 -> 2
14:48:23 Cluster one layer: 2 -> 1
14:48:23 Done (0.16s)
14:48:54 Knowledge graph extraction error:**ERROR**: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o-mini in organization org-XXXXXXXXXXXXXXXXXXXXXXXX on tokens per min (TPM): Limit 200000, Used 199481, Requested 2126. Please try again in 482ms. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
14:48:54 Knowledge graph extraction error:**ERROR**: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o-mini in organization org-XXXXXXXXXXXXXXXXXXXXXXXX on tokens per min (TPM): Limit 200000, Used 199833, Requested 2112. Please try again in 583ms. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

hi，same problem .how to solve the rate limit when use chat model is qwen. deploy by vllm. case： Retrieval testing, and enable Use knowledge graph

[KevinHuSh]

@sammichenVV export MAX_CONCURRENT_CHATS=2 Give it a try.