fanglioc

Server cannot run with 3080 Ti

4

While trying to deploy the server with: ```python -m ochat.serving.openai_api_server --model openchat/openchat_3.5```, I got the following error: ``` torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 224.00 MiB. GPU 0...