Tom

Yes, multinode is out of date. Sorry, I need to update it.

The recommended usage of ShardWriter is to write to local disk and then copy to the cloud. I'll add direct writing to the cloud as a possible enhancement.

Thanks; I'll try to reproduce it. It looks like there may be an unexpected filename in there somewhere. The Python code is a bit more forgiving.

I've added the ability to specify a handler. (Sorry it took so long.) However, having duplicate filenames really is a serious problem with the dataset, and I would recommend just...

Default is now USTAR_FORMAT, which should be more compact.

I'll have a look at a PR. Note that you can pass custom collate functions, so it's just a one liner anyway.

I think the source of this issue is that the keyboard map includes the normal keys ("a", "1", etc.). This causes them to be included as special multi-stroke sequences. It...

I think it would be good if this were documented better. Perhaps you can add an example to the home page (say, using 8bit bytes and auto placement) and improve...

Sorry for the long delay. WebDataset does not retain any samples unless you ask it to explicitly (e.g. in a shuffle buffer). In you case, just be sure not to...

Yes, depending on how you set up the pipeline, the pipeline will stop reading the current shard and start from the beginning of some shard at the next epoch. Furthermore,...