Zach Mueller

Launch the code using `python` as it mentions, not `accelerate launch`

Hi @byi8220, will be reviewing this in the next few days!

TBH I'm not sure if I like either. Let me work on this today and tommorow and see where I end up. Whichever solution winds up being similar to yours...

@byi8220 can you resolve the PR's and then I think we're okay to merge this.

As a final step, we likely want to update `save_state/load_state` to resume the dataloaders at this point.

> The tests highlighted one small thing though, the fact that to fully stop using a dataloader in the middle you have to call dataloader.end(), but this might just be...

as a next step I'll work on getting this working with `Accelerator.save_state`/`Accelerator.load_state` today

Thanks for your patience while I was out on holiday! Our general rule of thumb with Accelerate to keep things stable is the last 2 years of PyTorch releases, which...

cc @ArthurZucker for final :)

And in [transformers](https://github.com/huggingface/transformers/issues/29047) as well (in the future, please do not do this and just open in one. All of these were opened within minutes of each other).