Alliot Nagle comments

Repositories
Issues
Comments

Results 4 comments of


                                            Alliot Nagle

save_state not working correctly with FSDP on distributed setup

@pacman100 @muellerzr I'm having this issue as well, even after trying the `wait_for_everyone()` fix. As mentioned before, this likely has nothing to do with the `distributed_type` and instead seems to...

save_state not working correctly with FSDP on distributed setup

Thanks for the reply @pacman100 . Here is a script for @muellerzr that reproduces the error with my above Accelerate config: ```python import os import random import numpy as np...

Understanding the interplay between `ratio` and `iterative_size`

Hello again, I was reviewing the code and realized that I might not have understood your explanation as well as I thought. Just to make sure I understand: by using...

Understanding the interplay between `ratio` and `iterative_size`

Thank you for your response. I'm looking forward to the followup work!