FastChat
FastChat copied to clipboard
lm-sys
Error when loading Web UI
I met error when loading the UI , test_message passed
I followed the steps:
1.Launch the controller
2.Launch the model_worker
3. serve.test_message [ok]
4. Launch the Gradio web server
Errors shows in WEB UI, cannot interact with chatbot. Please advice,thanks
Here is the logs: controller.log 2023-04-25 07:00:52 | INFO | controller | args: Namespace(host='0.0.0.0', port=21001, dispatch_method='shortest_queue') 2023-04-25 07:00:52 | INFO | controller | Init controller 2023-04-25 07:00:52 | ERROR | stderr | INFO: Started server process [415983] 2023-04-25 07:00:52 | ERROR | stderr | INFO: Waiting for application startup. 2023-04-25 07:00:52 | ERROR | stderr | INFO: Application startup complete. 2023-04-25 07:00:52 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:21001 (Press CTRL+C to quit) 2023-04-25 07:01:45 | INFO | controller | Register a new worker: http://0.0.0.0:21002 2023-04-25 07:01:45 | INFO | controller | Register done: http://0.0.0.0:21002, {'model_names': ['vicuna-13b-v1.1'], 'speed': 1, 'queue_length': 0} 2023-04-25 07:01:45 | INFO | stdout | INFO: 127.0.0.1:56231 - "POST /register_worker HTTP/1.1" 200 OK 2023-04-25 07:02:15 | INFO | controller | Receive heart beat. http://0.0.0.0:21002 2023-04-25 07:02:15 | INFO | stdout | INFO: 127.0.0.1:6834 - "POST /receive_heart_beat HTTP/1.1" 200 OK 2023-04-25 07:02:45 | INFO | controller | Receive heart beat. http://0.0.0.0:21002 2023-04-25 07:02:45 | INFO | stdout | INFO: 127.0.0.1:21534 - "POST /receive_heart_beat HTTP/1.1" 200 OK
model_woker.log: 2023-04-25 07:01:16 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=21002, worker_address='http://0.0.0.0:21002', controller_address='http://0.0.0.0:21001', model_path='vicuna_data/vicuna-13b/', model_name='vicuna-13b-v1.1', device='cuda', num_gpus=2, max_gpu_memory=None, load_8bit=False, limit_model_concurrency=5, stream_interval=2, no_register=False) 2023-04-25 07:01:16 | INFO | model_worker | Loading the model vicuna-13b-v1.1 on worker c1b23a ... 2023-04-25 07:01:17 | INFO | stdout | init_kwargs {'torch_dtype': torch.float16, 'device_map': 'sequential', 'max_memory':{0: '20GiB', 1: '20GiB'}} Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] Loading checkpoint shards: 33%|████████████████████▎ | 1/3 [00:10<00:20, 10.07s/it] Loading checkpoint shards: 67%|████████████████████████████████████████▋ | 2/3 [00:19<00:09, 9.92s/it] Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████| 3/3 [00:26<00:00, 9.13s/it] Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████| 3/3 [00:26<00:00, 8.97s/it] 2023-04-25 07:01:45 | ERROR | stderr | 2023-04-25 07:01:45 | INFO | model_worker | Register to controller 2023-04-25 07:01:45 | ERROR | stderr | INFO: Started server process [416507] 2023-04-25 07:01:45 | ERROR | stderr | INFO: Waiting for application startup. 2023-04-25 07:01:45 | ERROR | stderr | INFO: Application startup complete. 2023-04-25 07:01:45 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:21002 (Press CTRL+C to quit) 2023-04-25 07:02:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-13b-v1.1']. Semaphore: None. global_counter:0 2023-04-25 07:02:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-13b-v1.1']. Semaphore: None. global_counter:0 2023-04-25 07:03:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-13b-v1.1']. Semaphore: None. global_counter:0 2023-04-25 07:03:45 | INFO | model_worker | Send heart beat. Models: ['vicuna-13b-v1.1']. Semaphore: None. global_counter:0 2023-04-25 07:04:15 | INFO | model_worker | Send heart beat. Models: ['vicuna-13b-v1.1']. Semaphore: None. global_counter:0
gradio_web_server.log 2023-04-25 07:18:38 | INFO | gradio_web_server | args: Namespace(host='0.0.0.0', port=5060, controller_url='http://0.0.0.0:21001', concurrency_count=10, model_list_mode='once', share='True', moderate='True') 2023-04-25 07:18:38 | INFO | gradio_web_server | Models: ['vicuna-13b-v1.1'] 2023-04-25 07:18:38 | INFO | gradio_web_server | Namespace(host='0.0.0.0', port=5060, controller_url='http://0.0.0.0:21001', concurrency_count=10, model_list_mode='once', share='True', moderate='True') 2023-04-25 07:18:39 | INFO | stdout | Running on local URL: http://0.0.0.0:5060
do you have gradio or controller log after your request?
my environment gradio 3.23.0
2023-04-27 02:02:23 | INFO | gradio_web_server | Namespace(host='0.0.0.0', port=None, controller_url='http://localhost:21001', concurrency_count=10, model_list_mode='once', share=False, moderate=False)
the shareandmoderate is different.
【Solved】, network configuration issue. Used un internal domain, need to change websocket configuration.
【Solved】, network configuration issue. Used un internal domain, need to change websocket configuration.
Can you tell me more information?
It needs to be configured in your web server (nginx) to proxy to websocket as follows
web socket http header config
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream vicuna_7860 {
server server1:7860;
}
server {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header REMOTE_ADD $remote_addr;
proxy_http_version 1.1;
location / {
proxy_pass http://vicuna_7860;
}
location /queue/join {
proxy_pass http://vicuna_7860;
proxy_redirect off;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
@kaifeng0502 thanks for the ideas
