data
data copied to clipboard
pytorch
A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.
### 🚀 The feature I want to apply the functionality of a already existing datapipe to only parts of my input. Below I listed some solutions. I would like to...
Hi TorchData team, I'm wondering how to access the state of the datapipe in the multi-processing context with DataLoader2 + MultiProcessingReadingService. When using no reading service, we can simply access...
- Added communication methods for state pickling in worker processes - The worker datapipes are briefly paused for the state to be gathered - The state is then gathered by...
### 🚀 The feature For context please read https://github.com/pytorch/data/issues/795 first. Iterating over MapDatapipes is currently inconsistent, we should find a way to resolve this. ### Motivation, pitch ```python source_dp =...
## 🚀 Feature We have many tests for existing DataPipes (both in PyTorch Core and TorchData). However, over time, they have become less organized. Moreover, as the [testing requirements](https://github.com/pytorch/pytorch/wiki/DataPipes-testing-requirements) expand,...
When running `python setup.py develop` (after having already gone through the installation) I get: error: urllib3 2.0.0a3 is installed but urllib3=1.21.1 is required by {'requests'} but in the requirements file...
### 🚀 The feature This is a customer request for inference, no synchronization is needed because no `all_reduce` is needed. And, normally inference doesn't require shuffling and requires high throughput....
### 🚀 The feature Add an `Adapter` for `pin_memory` feature, similar to enable/disable `shuffle`. ### Motivation, pitch This allows users to enable/disable/change device of `PinMemoryIterDataPipe` without making changes to the...
### 🐛 Describe the bug I used a local function in a pipeline which results in the warning: > UserWarning: Local function is not supported by pickle, please use regular...
MacOS continues to fail during intermittently during shutdown step. Before creating an issue in actions/setup-python let's try upgrading this to V5 Fixes #{issue number} ### Changes - -
