Jenny Zhang comments

Repositories
Issues
Comments

Results 1 comments of


                                            Jenny Zhang

13B Model Out of Memory with Single Node 8 A100 GPUs

What is the hardware and speed for the person fine-tuning the 20B parameter model above? I am interested in using GPT-NeoX-20B for fine-tuning. Would 2 servers of 8 A100s GPU...