infiniflow
[Question]: Multiple concurrent chats cause database connection error and can't update token usage error
Describe your problem
I am encountering a database connection error (peewee.InterfaceError: (0, '')) when multiple users chat with the same chatbot instance concurrently.
I have set up RagFlow using Docker, and everything works fine for single-user interactions. However, when running a stress test with multiple users (e.g., 50-100 concurrent sessions), the system frequently crashes with the following error:
025-02-20 21:34:37,340 INFO 14 172.18.0.6 - - [20/Feb/2025 21:34:37] "POST /api/v1/chats/ac3396acef3311efa6510242ac120006/completions HTTP/1.1" 200 -
2025-02-20 21:34:40,062 INFO 14 172.18.0.6 - - [20/Feb/2025 21:34:40] "POST /v1/conversation/set HTTP/1.1" 200 -
2025-02-20 21:34:40,084 INFO 14 172.18.0.6 - - [20/Feb/2025 21:34:40] "GET /v1/conversation/list?dialog_id=ac3396acef3311efa6510242ac120006 HTTP/1.1" 200 -
2025-02-20 21:34:40,191 ERROR 14 LLMBundle.encode_queries can't update token usage for 670c4e1eef3011efbe580242ac120006/EMBEDDING used_tokens: 2
2025-02-20 21:34:40,272 INFO 14 POST http://es01:9200/ragflow_670c4e1eef3011efbe580242ac120006/_search [status:200 duration:0.006s]
2025-02-20 21:34:40,326 INFO 14 POST http://es01:9200/ragflow_670c4e1eef3011efbe580242ac120006/_search [status:200 duration:0.006s]
2025-02-20 21:34:50,616 INFO 14 HTTP Request: POST http://192.168.1.55:11434/api/chat "HTTP/1.1 200 OK"
2025-02-20 21:34:50,617 ERROR 14 (0, '')
Traceback (most recent call last):
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 562, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 843, in _execute_command
raise err.InterfaceError(0, "")
pymysql.err.InterfaceError: (0, '')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "/ragflow/api/utils/api_utils.py", line 303, in decorated_function
return func(*args, **kwargs)
File "/ragflow/api/apps/sdk/session.py", line 156, in chat_completion
for ans in rag_completion(tenant_id, chat_id, **req):
File "/ragflow/api/db/services/conversation_service.py", line 156, in completion
ConversationService.update_by_id(conv.id, conv.to_dict())
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3128, in inner
return fn(*args, **kwargs)
File "/ragflow/api/db/services/common_service.py", line 109, in update_by_id
num = cls.model.update(data).where(cls.model.id == pid).execute()
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2011, in inner
return method(self, database, *args, **kwargs)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2082, in execute
return self._execute(database)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2600, in _execute
cursor = database.execute(self)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3299, in execute
return self.execute_sql(sql, params)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3289, in execute_sql
with __exception_wrapper__:
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3059, in __exit__
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3291, in execute_sql
cursor.execute(sql, params or ())
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 562, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 843, in _execute_command
raise err.InterfaceError(0, "")
peewee.InterfaceError: (0, '')
Here is my test script
import requests
import concurrent.futures
import time
import random
CHAT_ID = "XXXXXXX" # All dummy users share the same assistant
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {API_KEY}"
}
def create_session():
""" Create a new session for each user """
session_url = f"{BASE_URL}/chats/{CHAT_ID}/sessions"
session_data = {"name": f"test_session_{random.randint(1000, 9999)}"}
response = requests.post(session_url, json=session_data, headers=headers)
return response.json()["data"]["id"] if response.status_code == 200 else None
def chat_with_bot(session_id):
""" Simulate user chat """
chat_url = f"{BASE_URL}/chats/{CHAT_ID}/completions"
chat_data = {
"question": "What are the hospital's operating hours?",
"session_id": session_id,
"stream": False
}
response = requests.post(chat_url, json=chat_data, headers=headers)
return response.json()
# Simulating 100 users chatting at the same time
with concurrent.futures.ThreadPoolExecutor(max_workers=50) as executor:
futures = [executor.submit(chat_with_bot, create_session()) for _ in range(100)]
concurrent.futures.wait(futures)
I'm just wonderring if I should create a separate assistant (CHAT_ID) per user to avoid conflicts? Any other suggestions?
Thx a lot!
我也有这样的问题，我是在两个用户或以上同时使用同一个chatid时，就不输出了
我是报了这个错误： 2025-02-21 10:42:21,959 ERROR 26 LLMBundle.similarity can't update token usage for cdec426eac9711ef8fd30242ac130006/RERANK used_tokens: 60004
我也有这样的问题，我是在两个用户或以上同时使用同一个chatid时，就不输出了
我是报了这个错误： 2025-02-21 10：42：21,959 错误 26 LLMBundle.similarity 无法更新 cdec426eac9711ef8fd30242ac130006/RERANK used_tokens的令牌用法：60004
我把RERANK模型取消掉就可以正常使用了，有没有办法可以让rerank支持并发呢？
Probably there's something wrong with the database. I just found when the error below occurs,
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 843, in _execute_command
raise err.InterfaceError(0, "")
restart docker ragflow-mysql , and it works fine again.
Did you make it work for 100 users? what was your expectation and final result?
Also, did it bring the Ragflow to its knees? what is your hardware configuration?
Are you running on the cloud or local machine. I am looking for a template to deploy to GCP or AWS into k8s managed cluster. Do you have one?
