torchdistx
torchdistx copied to clipboard
pytorch
Torch Distributed Experimental
feat(dmodule): support parallelized dtensor init feat(dtensor): support for query random op feat(dtensor): support deferred init on device **What does this PR do? Please describe:** hello, i add some support for...
i found that when use torchdistx with deferred_init it will hang as cuda memory copy
This PR should is made to land together with this PyTorch refactor PR: https://github.com/pytorch/pytorch/pull/118846
**What does this PR do:** Fixing compatibility issues with PyTorch versions above 2.1.0, corresponding issue: #79 **Does your PR introduce any breaking changes? If yes, please list them:** `ProxyVariableHooks::basic_autograd_not_implemented_fallback`
As @stas00 mentions, AnyPrecisionOptimizer documentation should exist here: https://pytorch.org/torchdistx/latest/, otherwise users will not be aware of new features and may not try them out.
**Describe the bug:** $ cmake --build build [ 12%] Building CXX object src/cc/torchdistx/CMakeFiles/torchdistx.dir/deferred_init.cc.o In file included from /usr/include/c++/11/memory:76, from /home/ubuntu/.local/lib/python3.8/site-packages/torch/include/c10/util/C++17.h:8, from /home/ubuntu/.local/lib/python3.8/site-packages/torch/include/c10/util/string_view.h:4, from /home/ubuntu/.local/lib/python3.8/site-packages/torch/include/c10/util/StringUtil.h:6, from /home/ubuntu/.local/lib/python3.8/site-packages/torch/include/c10/util/Exception.h:5, from /home/ubuntu/.local/lib/python3.8/site-packages/torch/include/c10/core/DispatchKeySet.h:3, from /home/ubuntu/.local/lib/python3.8/site-packages/torch/include/c10/core/impl/LocalDispatchKeySet.h:3,...
**What does this PR do? Please describe:** Add script to install `cudnn` with `CUDA 11.7`. Fixes #{issue number}: N.A. **Does your PR introduce any breaking changes? If yes, please list...
There are some great features in this project, but it seems that they won't be supported anymore (since the PyTorch Foundation was spun off?), since the project can't be built...
Updates C++ code based on BC-breaking change https://github.com/pytorch/pytorch/pull/92169
Hi! **Describe the bug:** I am trying to build `torchdistx` from source following the instructions in the [readme](https://github.com/pytorch/torchdistx#from-source). Basically, I am running - ``` pip install --upgrade -r requirements.txt -r...
