Benjamin Badger

Thanks very much for reviewing everyone! @muellerzr @SunMarc your suggestions are much appreciated, I have gone ahead and implemented pretty much all of them. A test for `save_pretrained` applied to...

@SunMarc sounds good to me! Thanks for the review and for giving pointers on where the method would live as well. I will open a PR in `accelerate` for adding...

@ArthurZucker thanks much for the review! Just to make sure I understand correctly, are you referring to adding compatibility for an offloaded model to be saved when it does not...

We should be good to go for saving models too large to fit on cpu memory (although I am still cleaning up the code and adding to the test). The...

Code has now been refactored and the test is updated, all are passing now. I have run the test externally with small `max_shard_size` to confirm that the model only loads...

Went ahead and added cpu memory checks via `psutil` to the test. The idea is that the `model.save_pretrained` function will onload no more memory than `max_shard_size` from the disk to...

No problem, sounds good to me! I have also performed tests on disk storage used and there does not appear to be any change in storage usage compared to the...

After some re-examination this approach seems fairly robust and straightforward with the exception of the sharded memory allocation. It appears to me that the current block allocation using `id_tensor_storage()` will...

Added a warning for the user to check that the free memory exceeds the `shard_size`, which may not be the case for `device_map="auto" and would cause saving to fail otherwise...

@ ArthurZucker no problem, thanks for continuing with this!