[Bug]: Agent Chat Error - openai.NotFoundError

[meetcharan-practice]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (Language Policy).
• [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• [x] Please do not modify this template :) and fill in all the required fields.

RAGFlow workspace code commit ID

#11601 #11666

RAGFlow image version

NA

Other environment information

Actual behavior

I'm encountering the following error when trying to use the Agent Chat. I pulled the latest source a few hours ago.

Traceback (most recent call last): File "/ragflow/rag/llm/chat_model.py", line 750, in async_chat return await self._async_chat(history, gen_conf, **kwargs) File "/ragflow/rag/llm/chat_model.py", line 734, in _async_chat response = await self.async_client.chat.completions.create(model=self.model_name, messages=history, **gen_conf, **kwargs) File "/ragflow/.venv/lib/python3.10/site-packages/openai/resources/chat/completions/completions.py", line 2672, in create return await self._post( File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1794, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1594, in request raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-04 12:03:08,464 ERROR 22 async base giving up: ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-04 12:03:08,467 ERROR 22 Wrong JSON argument format in LLM ReAct response: List should be returned, but {'error': {'code': '404', 'message': 'Resource not found'}} Traceback (most recent call last): File "/ragflow/agent/component/agent_with_tools.py", line 348, in _react_with_tools_streamly_async raise TypeError(f"List should be returned, but {functions}") TypeError: List should be returned, but {'error': {'code': '404', 'message': 'Resource not found'}}

Expected behavior

No response

Steps to reproduce

Create a Docs QA Agent and configure a Retrieval Tool where the knowledge base is selected.

Additional information

No response

[dosubot[bot]]

The 404 error from OpenAI in Agent Chat almost always means the model name or factory in your configuration is invalid or not authorized for your API key. Recent changes in RAGFlow have affected how model names, factories, and base URLs are handled, so misconfiguration is a common cause—especially if the model name includes an unsupported suffix, a typo, or the '@factory' part is incorrect or missing. Each model should use its own base URL and be registered with the correct factory/provider; otherwise, the OpenAI API will return "Resource not found" (source).

To troubleshoot:

1. Check your Agent Chat configuration for the exact model name, factory, and baseUrl. Make sure the model name matches what your OpenAI account supports (e.g., 'gpt-3.5-turbo' or 'gpt-4'), and that the factory is set to 'OpenAI' or 'OpenAI-API-Compatible' as appropriate.
2. Inspect your tenant's llm_id and the TenantLLM table for any extra suffixes (like '@factory' or '___OpenAI-API') that might be causing issues.
3. Confirm your API key and endpoint are correct and that your OpenAI account has access to the requested model.
4. Try switching to a known working model/factory combo to isolate the problem.

If you share your current model name, factory, and baseUrl (redacting sensitive info), I can help pinpoint the misconfiguration. Recent code changes mean each model must use its own baseUrl and factory, so double-check those settings (source).

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[meetcharan-practice]

@dosu - The same model works under Chat but fails in Agent Chat. It was functioning correctly until yesterday, before I pulled the latest changes from the repository today.

Model Name: gpt-4o

[dosubot[bot]]

This 404 error in Agent Chat after the latest update is almost certainly due to a change in how model names, factories, or base URLs are resolved and passed to the OpenAI API. The recent async migration and refactoring (PR #11601) affected both chat and agent workflows, and several users have reported similar issues where Agent Chat fails with "Resource not found" while regular Chat works fine. The stack now relies on resolving model/factory/baseUrl triplets for each call, and any mismatch or misconfiguration—like an incorrect factory suffix, missing baseUrl, or a model name not authorized for your API key—will cause this error in Agent Chat, even if Chat works "source".

Check that your Agent Chat configuration for 'gpt-4o' matches the working Chat config exactly, including the factory and baseUrl. Make sure the model name does not have an extra '@factory' suffix or unsupported region, and that your API key is authorized for 'gpt-4o' under the selected factory. If you recently changed model settings, double-check the tenant's llm_id and the resolved model string in the database. If you use custom base URLs, confirm each model uses its own and not a default from another model "source".

If you still see the error, try switching to a known working model/factory combo to isolate the problem. This issue is actively being discussed and tracked, so further fixes may be incoming.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[yongtenglei]

Hi, @meetcharan-practice

Could you please configure your GPT-4o via OpenAI-Compatible factory rather than OpenAI? And let me know if it works for you?

Cheers.

[Magicbook1108]

I've made some test and nothing abnormal here.

[meetcharan-practice]

@Magicbook1108 - I believe you have the image with the latest source as of yesterday?

Traceback (most recent call last): File "/ragflow/rag/llm/chat_model.py", line 750, in async_chat return await self._async_chat(history, gen_conf, **kwargs) File "/ragflow/rag/llm/chat_model.py", line 734, in _async_chat response = await self.async_client.chat.completions.create(model=self.model_name, messages=history, **gen_conf, **kwargs) File "/ragflow/.venv/lib/python3.10/site-packages/openai/resources/chat/completions/completions.py", line 2672, in create return await self._post( File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1794, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1594, in request raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-05 11:35:29,993 ERROR 24 async base giving up: ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-05 11:35:29,995 ERROR 24 Wrong JSON argument format in LLM ReAct response: List should be returned, but {'error': {'code': '404', 'message': 'Resource not found'}} Traceback (most recent call last): File "/ragflow/agent/component/agent_with_tools.py", line 348, in _react_with_tools_streamly_async raise TypeError(f"List should be returned, but {functions}") TypeError: List should be returned, but {'error': {'code': '404', 'message': 'Resource not found'}}

[Magicbook1108]

Yeah Im the latest code.

[meetcharan-practice]

@yongtenglei - I am not able to configure under OpenAI-API-Compatible due to the following error.

hint : 102 Fail to access model(OpenAI-API-Compatible/gpt-4o).ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

My model is hosted in Azure. It is added like below.

[yongtenglei]

Hi, @meetcharan-practice

I am not quite sure. But could please try this patch and see if it solves this problem.

In rag/llm/chat_model.py:

Please let me know. Thank you!

[meetcharan-practice]

@yongtenglei – At this point, I’m not able to add the Azure OpenAI LLM model at all due to the errors shown below (both with and without the changes you suggested). This is with the latest source pulled today.

For reference, I tested the same LLM using a standalone Python script and it works fine. I also have another RAGFlow instance running an older version where it works without issues. This strongly indicates that the problem is caused by recent changes in the latest build.

2025-12-08 10:19:20 +0100] [27] [INFO] 127.0.0.1:50552 POST /v1/llm/add_llm 1.1 200 176 712670 2025-12-08 10:19:29,758 INFO 27 [HISTORY][ { "role": "user", "content": "Hello! How are you doing!" } ]

2025-12-08 10:19:30,032 ERROR 27 OpenAI async completion Traceback (most recent call last): File "/ragflow/rag/llm/chat_model.py", line 483, in async_chat return await self._async_chat(history, gen_conf, **kwargs) File "/ragflow/rag/llm/chat_model.py", line 467, in _async_chat response = await self.async_client.chat.completions.create(model=self.model_name, messages=history, **gen_conf, **kwargs) File "/ragflow/.venv/lib/python3.10/site-packages/openai/resources/chat/completions/completions.py", line 2672, in create return await self._post( File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1794, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1594, in request raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-08 10:19:30,035 ERROR 27 async base giving up: ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-08 10:19:30,036 ERROR 27 Fail to access model(Azure-OpenAI/gpt-4o).ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} NoneType: None [2025-12-08 10:19:30 +0100] [27] [INFO] 127.0.0.1:58984 POST /v1/llm/add_llm 1.1 200 176 478688

I was able to add the LLM after enabling the vision support toggle, but I'm now encountering the following error when trying to use it in the chat window.

2025-12-08 14:05:28,490 ERROR 384 Model <rag.llm.cv_model.AzureGptV4 object at 0x75b4e5c440a0> does not implement async_chat or async_chat_with_tools Traceback (most recent call last): File "/ragflow/api/apps/conversation_app.py", line 224, in stream async for ans in async_chat(dia, msg, True, **req): File "/ragflow/api/db/services/dialog_service.py", line 436, in async_chat questions[-1] += keyword_extraction(chat_mdl, questions[-1]) File "/ragflow/rag/prompts/generator.py", line 180, in keyword_extraction kwd = chat_mdl.chat(rendered_prompt, msg[1:], {"temperature": 0.2}) File "/ragflow/api/db/services/llm_service.py", line 322, in chat return self._run_coroutine_sync(self.async_chat(system, history, gen_conf, **kwargs)) File "/ragflow/api/db/services/llm_service.py", line 319, in _run_coroutine_sync raise value File "/ragflow/api/db/services/llm_service.py", line 308, in runner result_queue.put((True, asyncio.run(coro))) File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/ragflow/api/db/services/llm_service.py", line 392, in async_chat raise RuntimeError(f"Model {self.mdl} does not implement async_chat or async_chat_with_tools") RuntimeError: Model <rag.llm.cv_model.AzureGptV4 object at 0x75b4e5c440a0> does not implement async_chat or async_chat_with_tools

[yongtenglei]

Hi, @meetcharan-practice

Thank you for your update. I found a post here. Could you please apply the patch I mentioned earlier, and make sure you have the correct api_version set in both RAGFlow and your Azure console? Let me know if it works.

For the issue with image2text models, I will provide another patch. RAGFlow is currently in an unstable state as it is transitioning from a sync to an async model. Thanks again for your willingness to keep updating and for your patience.

Cheers.

[meetcharan-practice]

@yongtenglei - I have verified using the patch you provided, but the issue still persists.

For comparison, I tested the same LLM with a standalone Python script and it works correctly. I also have another RAGFlow instance running an older version where everything functions as expected.

Please find the screenshots below for reference.

2025-12-09 10:51:05,291 INFO 18 [HISTORY][ { "role": "user", "content": "Hello! How are you doing!" } ] 2025-12-09 10:51:05,818 ERROR 18 OpenAI async completion Traceback (most recent call last): File "/ragflow/rag/llm/chat_model.py", line 483, in async_chat return await self._async_chat(history, gen_conf, **kwargs) File "/ragflow/rag/llm/chat_model.py", line 467, in _async_chat response = await self.async_client.chat.completions.create(model=self.model_name, messages=history, **gen_conf, **kwargs) File "/ragflow/.venv/lib/python3.10/site-packages/openai/resources/chat/completions/completions.py", line 2672, in create return await self._post( File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1794, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/ragflow/.venv/lib/python3.10/site-packages/openai/_base_client.py", line 1594, in request raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-09 10:51:05,820 ERROR 18 async base giving up: ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} 2025-12-09 10:51:05,822 ERROR 18 Fail to access model(Azure-OpenAI/STG-gpt-4o).ERROR: MODEL_ERROR - Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} NoneType: None [2025-12-09 10:51:05 +0100] [18] [INFO] 127.0.0.1:42814 POST /v1/llm/add_llm 1.1 200 176 690034

And please note that I am able to add the Embedding model with the same details except the model name.

Important Note: The same configuration works when I enable the support vision.

[meetcharan-practice]

@yongtenglei - FYI. We are still having this issue.

2025-12-12 09:55:28,340 ERROR 20 OpenAI async completion Traceback (most recent call last): File "/ragflow/rag/llm/chat_model.py", line 482, in async_chat return await self._async_chat(history, gen_conf, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ragflow/rag/llm/chat_model.py", line 466, in _async_chat response = await self.async_client.chat.completions.create(model=self.model_name, messages=history, **gen_conf, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[yongtenglei]

Hi, @meetcharan-practice

May I have your mail? I cannot reproduce without the key. If that is possible, may I have it for several hours and then you can revoke it anytime. What do you think?

[meetcharan-practice]

@yongtenglei - Sorry, it is not possible to share the key as it belongs to my company. But we can connect over the call if that helps. My email address is [email protected].

[yongtenglei]

Hi, @meetcharan-practice

Could please have a quick test to see if it works for you? #11934.

[meetcharan-practice]

@yongtenglei - It's working perfectly now! 🙂 Thank you so much for the fix, I really appreciate it.

[yongtenglei]

Hi, @meetcharan-practice, no worries. I am going to close this issue then.

Cheers.