Andrew comments

Repositories
Issues
Comments

Results 4 comments of


                                            Andrew

trafficstars

Can not run 13B inference model. After loading the ckpt, it just stoped and the gpus are still occupied.

Same problem!

Can not run 13B inference model. After loading the ckpt, it just stoped and the gpus are still occupied.

Seems a potential reason is not having enough GPU? Even though 2x RTX 4090 and RTX A5000 each with 24GB should be enough, I was able to run only on...

Anyone able to run 7B on google colab?

Update: able to run on google colab pro. Seems 12 GB of ram is the issue.

Anyone able to run 7B on google colab?

> It's possible to make it work on the free version. Since colab gives you more GPU VRAM than RAM, what you'll want to do is load the checkpoint into...