data

data copied to clipboard

torchdata not compatible with torch 2.3.0

3

### 🐛 Describe the bug ## Describe the bug `torchdata` does not work with torch 2.3.0 because `DILL_AVAILABLE` is not available where expected: ``` Python 3.12.2 (main, Feb 6 2024,...

mmdanziger

### 🚀 The feature title ### Motivation, pitch rstrip/lstrip can lead to unintended behaviors ### Alternatives _No response_ ### Additional context _No response_

gokulavasan

Update `ProtocolServer`'s exceptions to be more specific

1

Within `ProtocolServer` inside `dataloader2/communication/protocol.py`, the exceptions being raised inside are generic `Exception`. Ideally, we should change the exceptions to be more specific, such that they can be handled easily elsewhere....

NivekT

Shard StatefulDataLoader tests

3

Fixes #1256 ### Changes * Rename test_state_dict tests so they are easy to shard in CI Actions - -

andrewkho

MacOS state_dict tests in CI are failing during shutdown

2

### 🐛 Describe the bug MacOS tests of StatefulDataLoader CI action fail intermittently during shutdown. on Mac it also takes a lot longer than both windows and ubuntu to shut...

andrewkho

Add message about s3 iterdata pipes deprecation

3

Adding deprecation message about S3 IterDataPipes and pointing to https://github.com/awslabs/s3-connector-for-pytorch. Cc @jamesbornholt, @dnauti

gokulavasan

Adding state parser utility that can be used for retrieving and modifying worker states

1

Summary: Allows users to get the worker states out of the state dict. It can be used if users want to modify the state offline. Starts off with very preliminary...

gokulavasan

State_dict on dataset seems to be called more often than expected

2

### 🐛 Describe the bug Consider the following code: ``` class DatasetStateIterable(torch.utils.data.IterableDataset, Stateful): def __init__(self, length): self.length = length def __iter__(self): return iter(list(range(self.length))) def state_dict(self): print("Calling state dict") return {"key":...

gokulavasan

### 🐛 Describe the bug Google Drive redirects users to a Virus Warining page, when the file size is large and it cannot scan for virus. This results in GDriveReaderDataPipe...

huaixv

Enable Append Mode in SaverIterDataPipe

1

### 🚀 The feature Currently, Saver only allows write mode and only users to choose byte vs text mode. It might be useful to allow the flexibility to append to...

rravu3