Srinivasan Iyer

icon indicating a website link http://sriniiyer.github.io icon indicating an email [email protected]

icon company University of Washington icon location Seattle, WA icon location Staff Research Scientist at FAIR

Results 5 comments of Srinivasan Iyer

Error when run reshard_fsdp on opt-IML 30b for inference

Could you share the motivation and the command? Thanks!

Error when run reshard_fsdp on opt-IML 30b for inference

@Charles-ux-bit We are working on uploading it to huggingface and it will be available soon, hopefully in a few days. An alternative that works right now is to use the...

Error when run reshard_fsdp on opt-IML 30b for inference

@xiangjjj I need the exact command you folks are trying to run - and what you are trying to achieve - in order to unblock you.

load_checkpoint_and_dispatch OOMs

> You cannot send your model to `accelerator.prepare` if using `device_map="auto"` (as the model will be split across GPUs already). Looks like load_state_dict is trying to put the entire model...

load_checkpoint_and_dispatch OOMs

@sgugger The from_pretrained works fine, so if I had saved my checkpoint using save_pretrained, everything is great. Unfortunately, I saved the state_dict, so when I load the state_dict, it OOMs...