accelerate icon indicating copy to clipboard operation
accelerate copied to clipboard

Only wrap modules in DDP if they require grad

Open samuelstevens opened this issue 3 years ago • 1 comments

Like the title says. Fixes #760.

There is just such a check in DistributedDataParallel, which I do the opposite of.

I would like to add a test so that this feature doesn't regress, but it's not clear to me how to add a test.

I could add a module with only buffers and add it to the prepare() call on line 78, then check that the device is the accelerator's device.

https://github.com/huggingface/accelerate/blob/693d46826e32507376d44f99967df4710886c984/tests/test_kwargs_handlers.py#L73-L79

samuelstevens avatar Oct 14 '22 18:10 samuelstevens

The documentation is not available anymore as the PR was closed or merged.