guagua comments

Repositories
Issues
Comments

Results 3 comments of


                                            guagua

CUDA OOM during ckpt saving for Llama2-70b

Hi @HamidShojanazeri , I also got the same OOM error when using 8xH100s. I'm using the transformer `'4.41.0.dev0'` and build the llama-recipes from the source. I observed this error will...

OOM when saving model, seems the clause "FSDP(model,...)" will cause the OOM.

Hi @HamidShojanazeri , I also got the same OOM error when using 8xH100s. I'm using the transformer '4.41.0.dev0' and build the llama-recipes from the source. I observed this error will...

OOM when saving model, seems the clause "FSDP(model,...)" will cause the OOM.

Hi @HamidShojanazeri , thank you for the reply. I was using lora to finetune the 70B llama3 model and ran into this OOM issue, that's why I thought this might...