Brendan Folie comments

Results 15 comments of


                                            Brendan Folie

[RFC] Improved coverage for native distributed collective operations

All collectives have been implemented in eager mode. Not all work in compiled mode but we have the same coverage as PT native does.

[RFC] Enable DTensor SPMD APIs with XLA SPMD Backend

> XLA doesn't have an explicit control on resharding a tensor, but this can be achieved by simply cloning the existing tensor and do a `mark_sharding` on new tensor Is...

[RFC] Enable DTensor SPMD APIs with XLA SPMD Backend

Thanks @rpsilva-aws, yes that is the discussion I was thinking of

[RFC] Enable DTensor SPMD APIs with XLA SPMD Backend

Thanks for this proposal and context @fhaolinaws. As we've discussed I think this is a good approach. It improves the user experience, provides both collective-based and XLA-based options, and should...

Adding functionality for metadata communication across hosts

The approach seems good. However I don't think this should be merged until the corresponding PyTorch PR has been merged and these tests can be activated.