FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.

Open 107064547 opened this issue 1 year ago • 3 comments

NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.

(error_code: 50003, HTTPConnectionPool(host='localhost', port=21002): Read timed out.) 每次问第二个问题都会出现这个错误提示

107064547 avatar May 29 '23 11:05 107064547

更新一下代码，新版本已经将 WORKER_API_TIMEOUT 这个参数从启动命令获取，没有设置默认100秒，老版本默认是20秒。也可以在老版本的constants.py文件中修改WORKER_API_TIMEOUT这个参数。 这个超时的一般原因是硬件水平较低或者网络较差。

swmvp avatar May 31 '23 02:05 swmvp

更新直接使用pip3 install fschat？发自我的 iPad在 2023年5月31日，10:57，swmvp @.***> 写道： 更新一下代码，新版本已经将 WORKER_API_TIMEOUT 这个参数从启动命令获取，没有设置默认100秒，老版本默认是20秒。也可以在老版本的constants.py文件中修改WORKER_API_TIMEOUT这个参数。 这个超时的一般原因是硬件水平较低或者网络较差。

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

107064547 avatar May 31 '23 04:05 107064547

更新一下FastChat项目代码（版本），不是fschat包

swmvp avatar Jun 09 '23 01:06 swmvp

。。。我第一问题就报错，都是同一台机部署的 超时什么鬼 ，看log也没发现什么问题 ， 用的FastChat 最新代码2fbfcbc4b72670f6e4d4833abf4446c6e0fd6950

python -m fastchat.serve.controller

python -m fastchat.serve.model_worker --model-path lmsys/vicuna-33b-v1.3 --load-8bit


python -m fastchat.serve.gradio_web_server

2023-08-31 10:59:35 | INFO | gradio_web_server | Models: []
2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:43: UserWarning: You have unused kwarg parameters in Dropdown, please remove them: {'container': False}
2023-08-31 10:59:36 | ERROR | stderr |   warnings.warn(
2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: `height` is deprecated in `Interface()`, please use it within `launch()` instead.
2023-08-31 10:59:36 | ERROR | stderr |   warnings.warn(value)
2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:43: UserWarning: You have unused kwarg parameters in Textbox, please remove them: {'container': False}
2023-08-31 10:59:36 | ERROR | stderr |   warnings.warn(
2023-08-31 10:59:36 | INFO | stdout | Running on local URL:  http://0.0.0.0:7860
2023-08-31 10:59:36 | INFO | stdout | 
2023-08-31 10:59:36 | INFO | stdout | To create a public link, set `share=True` in `launch()`.
 14%|█▍        | 1/7 [00:26<02:37, 26.22s/it]
 29%|██▊       | 2/7 [00:46<01:54, 22.83s/it]
 43%|████▎     | 3/7 [00:57<01:09, 17.48s/it]
 57%|█████▋    | 4/7 [01:17<00:54, 18.18s/it]
 71%|███████▏  | 5/7 [01:36<00:37, 18.57s/it]
 86%|████████▌ | 6/7 [01:54<00:18, 18.54s/it]
100%|██████████| 7/7 [02:13<00:00, 18.73s/it]
100%|██████████| 7/7 [02:13<00:00, 19.13s/it]
2023-08-31 11:01:38 | ERROR | stderr | 
2023-08-31 11:01:38 | INFO | model_worker | Register to controller
2023-08-31 11:01:38 | INFO | controller | Register a new worker: http://localhost:21002
2023-08-31 11:01:38 | INFO | controller | Register done: http://localhost:21002, {'model_names': ['vicuna-33b-v1.3'], 'speed': 1, 'queue_length': 0}
2023-08-31 11:01:38 | INFO | stdout | INFO:     127.0.0.1:58054 - "POST /register_worker HTTP/1.1" 200 OK
2023-08-31 11:01:38 | ERROR | stderr | INFO:     Started server process [47158]
2023-08-31 11:01:38 | ERROR | stderr | INFO:     Waiting for application startup.
2023-08-31 11:01:38 | ERROR | stderr | INFO:     Application startup complete.
2023-08-31 11:01:38 | ERROR | stderr | INFO:     Uvicorn running on http://localhost:21002 (Press CTRL+C to quit)
2023-08-31 11:02:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:02:23 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:02:23 | INFO | stdout | INFO:     127.0.0.1:37040 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:02:24 | INFO | gradio_web_server | load_demo. ip: 58.248.180.226. params: {}
2023-08-31 11:02:34 | INFO | gradio_web_server | load_demo. ip: 58.248.180.226. params: {}
2023-08-31 11:02:43 | INFO | gradio_web_server | add_text. ip: 58.248.180.226. len: 2
2023-08-31 11:02:43 | INFO | gradio_web_server | bot_response. ip: 58.248.180.226
2023-08-31 11:02:43 | INFO | stdout | INFO:     127.0.0.1:54646 - "POST /get_worker_address HTTP/1.1" 200 OK
2023-08-31 11:02:43 | INFO | gradio_web_server | model_name: , worker_addr: 
2023-08-31 11:03:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:03:08 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:03:08 | INFO | stdout | INFO:     127.0.0.1:47400 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:03:16 | INFO | gradio_web_server | load_demo. ip: 58.248.180.226. params: {}
2023-08-31 11:03:20 | INFO | gradio_web_server | add_text. ip: 58.248.180.226. len: 5
2023-08-31 11:03:20 | INFO | gradio_web_server | bot_response. ip: 58.248.180.226
2023-08-31 11:03:20 | INFO | stdout | INFO:     127.0.0.1:55472 - "POST /get_worker_address HTTP/1.1" 200 OK
2023-08-31 11:03:20 | INFO | gradio_web_server | model_name: , worker_addr: 
2023-08-31 11:03:27 | INFO | gradio_web_server | clear_history. ip: 58.248.180.226
2023-08-31 11:03:34 | INFO | gradio_web_server | add_text. ip: 58.248.180.226. len: 8
2023-08-31 11:03:34 | INFO | gradio_web_server | bot_response. ip: 58.248.180.226
2023-08-31 11:03:34 | INFO | stdout | INFO:     127.0.0.1:35528 - "POST /get_worker_address HTTP/1.1" 200 OK
2023-08-31 11:03:34 | INFO | gradio_web_server | model_name: , worker_addr: 
2023-08-31 11:03:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:03:53 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:03:53 | INFO | stdout | INFO:     127.0.0.1:37284 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:04:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:04:38 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:04:38 | INFO | stdout | INFO:     127.0.0.1:42878 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:05:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:05:23 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:05:23 | INFO | stdout | INFO:     127.0.0.1:42588 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:06:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:06:08 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:06:08 | INFO | stdout | INFO:     127.0.0.1:55096 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:06:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:06:53 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:06:53 | INFO | stdout | INFO:     127.0.0.1:48488 - "POST /receive_heart_beat HTTP/1.1" 200 OK
2023-08-31 11:07:39 | INFO | model_worker | Send heart beat. Models: ['vicuna-33b-v1.3']. Semaphore: None. call_ct: 0. worker_id: 00452e2d. 
2023-08-31 11:07:39 | INFO | controller | Receive heart beat. http://localhost:21002
2023-08-31 11:07:39 | INFO | stdout | INFO:     127.0.0.1:60876 - "POST /receive_heart_beat HTTP/1.1" 200 OK

631068264 avatar Aug 31 '23 03:08 631068264

机器配置贴出来

swmvp avatar Aug 31 '23 03:08 swmvp

NVIDIA A100 80GB * 4 GiB Mem : 251.902 total, 80.452 free, 96.438 used, 75.012 buff/cache

631068264 avatar Aug 31 '23 03:08 631068264

你的硬件非常土豪。 2``` 023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:43: UserWarning: You have unused kwarg parameters in Dropdown, please remove them: {'container': False} 2023-08-31 10:59:36 | ERROR | stderr | warnings.warn( 2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: height is deprecated in Interface(), please use it within launch() instead. 2023-08-31 10:59:36 | ERROR | stderr | warnings.warn(value) 2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:43: UserWarning: You have unused kwarg parameters in Textbox, please remove them: {'container': False} 2023-08-31 10:59:36 | ERROR | stderr | warnings.warn(

上面这些日志报错处理一下。

2023-08-31 11:01:38 | ERROR | stderr | INFO: Started server process [47158] 2023-08-31 11:01:38 | ERROR | stderr | INFO: Waiting for application startup. 2023-08-31 11:01:38 | ERROR | stderr | INFO: Application startup complete. 2023-08-31 11:01:38 | ERROR | stderr | INFO: Uvicorn running on http://localhost:21002 (Press CTRL+C to quit)

看这些日志 你的服务是启动了，但是有错误。

swmvp avatar Aug 31 '23 03:08 swmvp

你的硬件非常土豪。 2``` 023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:43: UserWarning: You have unused kwarg parameters in Dropdown, please remove them: {'container': False} 2023-08-31 10:59:36 | ERROR | stderr | warnings.warn( 2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: height is deprecated in Interface(), please use it within launch() instead. 2023-08-31 10:59:36 | ERROR | stderr | warnings.warn(value) 2023-08-31 10:59:36 | ERROR | stderr | /data/home/yaokj5/anaconda3/envs/vicuna/lib/python3.10/site-packages/gradio/deprecation.py:43: UserWarning: You have unused kwarg parameters in Textbox, please remove them: {'container': False} 2023-08-31 10:59:36 | ERROR | stderr | warnings.warn(

上面这些日志报错处理一下。

2023-08-31 11:01:38 | ERROR | stderr | INFO: Started server process [47158] 2023-08-31 11:01:38 | ERROR | stderr | INFO: Waiting for application startup. 2023-08-31 11:01:38 | ERROR | stderr | INFO: Application startup complete. 2023-08-31 11:01:38 | ERROR | stderr | INFO: Uvicorn running on http://localhost:21002 (Press CTRL+C to quit)

看这些日志 你的服务是启动了，但是有错误。

我不太清楚怎么处理，而且上面那些不是warning吗

631068264 avatar Aug 31 '23 04:08 631068264 

2023-08-31 10:59:35 | INFO | gradio_web_server | Models: []

Seems like your LLM model has not been successfully loaded. LLM models need to be loaded before executing the gradio web server. Therefore, you need to add sleep command to ensure LLM models has successfully loaded. Here is the code modification.

nohup python -m fastchat.serve.controller --host 0.0.0.0 &
nohup python -m fastchat.serve.model_worker --model-path lmsys/vicuna-33b-v1.3 --load-8bit --host 0.0.0.0 &
sleep 40 
python -m fastchat.serve.gradio_web_server --host 0.0.0.0

The following is the log in the gradio server

2023-09-13 07:48:38 | INFO | gradio_web_server | Models: ['vicuna-33b-v1.3']
2023-09-13 07:48:38 | INFO | stdout | Running on local URL:  http://0.0.0.0:7860
2023-09-13 07:48:38 | INFO | stdout | 
2023-09-13 07:48:38 | INFO | stdout | To create a public link, set `share=True` in `launch()`.

Make sure you can see the model name in the square bracket

2023-09-13 07:48:38 | INFO | gradio_web_server | Models: ['vicuna-33b-v1.3']

Instead of

2023-09-13 07:48:38 | INFO | gradio_web_server | Models: []

paultsoi1014 avatar Sep 13 '23 07:09 paultsoi1014