data
data copied to clipboard
pytorch
A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.
### 🚀 The feature Currently RandomSampler, BatchSampler are patched here https://github.com/pytorch/data/blob/main/torchdata/stateful_dataloader/sampler.py#L134-L135 to make them stateful and work out of the box with StatefulDataLoader. It would be useful to consider making...
This RFC was re-created due to a problem with the original. Summary of comments from previous issue below. ### 🚀 The feature **TL;DR \-** We want to lean into **modular...
### 🚀 The feature We should include builds for Python 3.13 though listed as experimental in order to start testing against FT/NoGIL python ### Motivation, pitch NoGIL is going to...
### 🐛 Describe the bug CI for Ubuntu / Linux in stateful dataloader takes about 20 seconds, but for mac and windows takes about 30+ minutes. Not only burning a...
### 🚀 The feature State Dict utilities to inspect state ### Motivation, pitch We should provide some utilites for users to inspect and potentially modify the dataloader state. Currently the...
Please read through our [contribution guide](https://github.com/pytorch/data/blob/main/CONTRIBUTING.md) prior to creating your pull request. - Note that there is a section on requirements related to adding a new DataPipe. Fixes #{issue number}...
### 🐛 Describe the bug Nightly release has been failing for a couple of months, but we weren't aware. Figure out alerting on these failures https://github.com/pytorch/data/actions/runs/11031708385 ### Versions main
### 🐛 Describe the bug ```python dp = ... dp = dp.sharding_filter() rs = MultiProcessingReadingService(num_workers=4) dataloader = DataLoader2(dp, reading_service=rs) for _ in dataloader: pass dataloader.shutdown() ``` ```python Exception: Can not...
Fixes https://github.com/pytorch/data/issues/1310 advance request index just for skip disabled_pipe ### Changes - -
title Fixes #{issue number} ### Changes - -