MiniGPT-4 i have two 12GB GPU, and how to use them in model?

i have two 12GB GPU, and how to use them in model?

Open WakingHours-GitHub opened this issue 1 year ago • 4 comments

fact in here, i have two 12gb GPU, and i can use 13B model in theory, but there are no any note about how to use two GPU to inference, so now i've hit a wall.

Apr 23 '23 01:04 WakingHours-GitHub

I think that multi GPU might not be supported in one session. I would try starting 2 instances with different --gpu-id, so one with 0 one with --gpu-id 1, but in this case you will need 2x as much memory (normal ram). I have never tried this so not sure if it is possible, just my idea.

Apr 25 '23 10:04 Korner83

I have four Titan XP GPUs with 12 GB each, and they work fine for running the Vicuna 13B model. However, I'm having trouble running MiniGPT4 on them and can't find a way to enable multi-GPU support.

Apr 26 '23 14:04 ClarkWain

I have four Titan XP GPUs with 12 GB each, and they work fine for running the Vicuna 13B model. However, I'm having trouble running MiniGPT4 on them and can't find a way to enable multi-GPU support. yep, that is I whet say.

Apr 27 '23 09:04 WakingHours-GitHub

One more question, does anyone has pre-trained or fine-tuned the minigpt4 model with less than 16GB per GPU memory? I've tried to evaluate model, it seems the gpu memory is enough. But for training in this repo, the GPU 16GB is not enough for me in my local environment.

Apr 30 '23 19:04 Jason-fan20

@WakingHours-GitHub did you find a way to enable multi GPU inference?

Sep 19 '23 09:09 sushilkhadkaanon

MiniGPT-4 MiniGPT-4 copied to clipboard

i have two 12GB GPU, and how to use them in model?

MiniGPT-4
MiniGPT-4 copied to clipboard