data

data copied to clipboard

Add an example using a library such as Hugging Face datasets

2

### 🚀 The feature I think it would make sense to provide a "real-world" example for using the StatefulDataloader with a popular library such as Hugging Face datasets. For example,...

conceptofmind

Async Checkpointing with DCP and Stateful Dataloader

2

Hello, I was wondering what the recommended way was to use Async Checkpointing with the Stateful Dataloader? Does this seem correct: ```python from torchdata.stateful_dataloader import StatefulDataLoader from torch.distributed.checkpoint.state_dict import get_state_dict,...

conceptofmind