Sylvain Gugger

Failure seem unrelated. Yet rebasing on main to remove the failure linked to the Datasets release and re-laucnhing spurious tests would be helpful to ease everyone's mind :-)

Nope, you can go ahead and merge :-)

The next release of Transformers will be in a month roughly. In the meantime, you can install it from source.

I think the issue has been solved on the `huggingface_hub` side, as long as you use the latest version. Please let us know otherwise!

We also use the `modules` folder in the cache to put a `transformers_module` containing the code of dynamic models/tokenizers/pipelines (that have their code on the Hub).

Same comment as in https://github.com/huggingface/transformers/pull/19132 Those kinds of checks belong in your libraries directly, not Accelerate or Transformers as we can't constantly add/remove checks each time there is a new...

Accelerate was built when there was no join contextmanager and the fact each datalaoder returns the same number of samples in all processes is pretty ingrained in the library. We...

@Chris-hughes10 This is mostly used in the whole evaluation part of Accelerate (so `gather` and `gather_for_metrics`) as for training we don't really care (plus the dataloaders very often have `drop_last=True`...

Arf, GitHub changed the projects completely and I can't create an old one for Accelerate. Anyhow, here a few steps I see - [ ] Honor batch samplers with a...

@nbroad1881 You might need to close this PR and open a fresh one as GitHub shows a diff of 68 files (probably due to a rebase).