Erjia Guan

You might want to take a look at the project called torchx in https://pytorch.org/torchx/latest/examples_apps/index.html. This might provide you the required feature.

#429 This PR introduces the linter to check if there is a `shuffle` before each `sharding`. This should mainly be used by domain libraries to verify their implementation. Note: users...

> "Auto doc gen for RC release" - I believe we can add a GH action for that (i.e. whenever a push is made to the branch release/0.x.0, build a...

Nightly release is working for wheel. https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html But, it seems the conda token we have used to upload packages to `pytorch` channel cannot be used to upload them to `pytorch-nightly`...

Another bug for wheel/conda, the nightly version currently relies on date https://github.com/pytorch/data/blob/76733df6a8903bf95b938cf611355f05f5c2f821/packaging/pkg_helpers.bash#L93 But, the date on CI macos machine is not correct. I need to do a quick test.

Closing this PR as the corresponding Diff has been abandoned. A workaround has been accomplished internally

Just found a tool that might help us to identify what's wrong with `StreamWrapper`. https://refcycle.readthedocs.io/en/latest/ This tool can help us to visualize the refcount of nested objects.

Just a note: `MultiProcessingReadingService` (`MPRS`) is a temporary reading service. We will change it to the the `PrototypeMultiProcessingReadingService`. And, for `prefetch_factor`, we might provide it as `DataPipes` to let users...

Could you please open a PR in PyTorch core by creating the validation function first? - [`Mapper`](https://github.com/pytorch/pytorch/blob/77b6885a225db0e4dfe331d3e98a5a456c672c2c/torch/utils/data/datapipes/iter/callable.py#L17) - [`Filter`](https://github.com/pytorch/pytorch/blob/dd620c45752ab746bdd6bc47e8a2425a2f79a702/torch/utils/data/datapipes/iter/selecting.py#L14) Then, TorchData can import such validation function from PyTorch Core and...

You can put it in https://github.com/pytorch/pytorch/blob/master/torch/utils/data/datapipes/utils/common.py