Will Constable

Results 116 comments of Will Constable

Torch Distributed Pipelinging API datatype mismatch

> Q3: The issue here seems to be that pipeline stage is saving forward outputs and passing those into backward, but it makes the assumption that the forward outputs are...

move pp tests

> why the move? I think we discussed at one point that it's not clear where to put the tests that focus on composing multiple things together, and as a...

[Draft][Pipelining] Implement compute to comm lowering

@pytorchbot merge

[Draft][Pipelining] Implement compute to comm lowering

@pytorchbot merge

[Draft][Pipelining] Implement compute to comm lowering

@pytorchbot merge

[Pipelining] add UNSHARD/RESHARD actions and test helpers

FYI I split the original PR so this one is just focusing on the refactor of _Action class. Then i add UNSHARD/RESHARD stuff in the next PR.

[Pipelining] add UNSHARD/RESHARD actions and test helpers

@pytorchbot merge

The ProcessGroupNCCL is not being destructed

I wonder if you could use torchrun to launch your job? Also, you may want to look at our unit test helpers which launch and join processes inside the base...

[Pipelining] Add to/from CSV format and improved __repr__

@pytorchbot merge

[Pipelining] Add to/from CSV format and improved __repr__

ok, this needs to be fixed. Ill put a merge cmd on the PR right before this and then work on trying to fix this PR docs