Chris (Tu) NGUYEN comments

Repositories
Issues
Comments

Results 1 comments of


                                            Chris (Tu) NGUYEN

fp32 Full Training seems to be taking a lot of memory

Hi @rohan-varma , I'm also having the same problem with full fine-tuning (fp32) on 8xV100 (32GB) - activation checkpointing enabled - FSDP: FULL_SHARD - batch_size = 1 Training can still...