Model running on only 2 GPU's even when 4 GPU's are specified

[SupreethRao99]

trafficstars

Hello,

I want to run fschat on all 4 of my GPU's but even when I specify to use all 4 GPU's the model is still being run on 2 of them. Code I used.

python3 -m fastchat.serve.model_worker --host 0.0.0.0 --port 8999 --worker-address http://0.0.0.0:8999 --controller-address http://0.0.0.0:8998 --model-path ./vicuna-13b --num-gpus 4

Nvidia-smi logs