data
data copied to clipboard
pytorch
A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.
### 🐛 Describe the bug Hi This bug is following the conversation on [discuss.pytorch.org](https://discuss.pytorch.org/t/using-a-google-cloud-storage-bucket-for-dataset/146253/14?u=sephib) When running the following code in a jupyter kernel - the fs.protocol is not consistent To...
### 🚀 The feature There are several options to disable specific linters. Option 1. Disable with `linter-ignore: code` Pros: - Similar to known syntax of various linters Cons: - Need...
### 🐛 Describe the bug During the work to fixing the problem with unhashable DataPipe in https://github.com/pytorch/pytorch/pull/80509, I find this test is broken: https://github.com/pytorch/pytorch/blob/e266bea79395399d60bd3c684545f69ae6900236/test/test_datapipe.py#L2307-L2349 The failure is: `TypeError: cannot pickle...
### 🚀 The feature This issue proposes the addition of a linter for DataPipes and DataLoader2. The linter can analyze the graph of DataPipes and input arguments to DataLoaderV, and...
For user dev and onboarding experience of the data component, we will provide examples, tutorials, up-to-date documentations as well as the operational support. We added a simple train loop example....
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #652 * __->__ #562 Differential Revision: [D40146676](https://our.internmc.facebook.com/intern/diff/D40146676)
This issue is generated from the TODO line https://github.com/pytorch/data/blob/2f29adba451e1b87f1c0c654557d9dd98673fdd8/torchdata/datapipes/iter/util/cacheholder.py#L236 cc @VitalyFedyunin
This issue is generated from the TODO line https://github.com/pytorch/data/blob/2f29adba451e1b87f1c0c654557d9dd98673fdd8/torchdata/datapipes/iter/util/tfrecordloader.py#L90
This issue is generated from the TODO line https://github.com/pytorch/data/blob/2f29adba451e1b87f1c0c654557d9dd98673fdd8/torchdata/datapipes/iter/util/tfrecordloader.py#L46