anshul-si

[torchtitan][replicate] experimenting new replicate integration with torchtitan

6

**Summary:** During this experiment to integrate the new replicate function into torchtitan, I used https://github.com/pytorch/pytorch/pull/162021, which has not been landed. However, since this is more about making replicate more efficient...

[dtensor][partial] fixes unnecessary redistributions when subtracting two partial tensors

1

**Summary:** Currently, whenever we subtract two partial dtensors, we redistribute since linearity is -1 for aten.sub.tensor. However, this is an unnecessary redistribution that can be avoided in similar ways to...

[dtensor][partial] fixed adding scalar to any Partial

1

**Summary:** When we add a scalar to Partial dtensor, we don't redistribute causing us to add the scalar to each part of the partial dtensor as reported in https://github.com/pytorch/pytorch/issues/149768, https://github.com/pytorch/pytorch/issues/163193....

[dtensor][partial] redistributes _NormPartial to replicate for necessary pointwise_ops

1

**Summary:** While linearities are correct for regular Partial tensors, they don't apply to NormPartial tensors. We fix this by redistributing for all ops necessary when the placement is NormPartial. The...

[dtensor][partial] preventing redistribution when linearity is 0

1

**Summary:** When users multiply a partial dtensor by a scalar, we should keep the output as a partial dtensor. However, we incorrectly force the partial to be redistributed to replicate...