Add writer.from_iterable_dataset for writing IterableDatasets to .beton

Open ericjang opened this issue 3 years ago • 1 comments

The current normalizer transform when operating on GPU tensors uses cupy, which assumes device=0 unless otherwise stated. This results in a bug where trying to run ffcv pipelines with Normalize on any other gpu causes an error to be raised claiming that the tensors are on different devices.

Jun 11 '22 20:06 ericjang

Has it been fixed? I also encountered this bug today

Sep 23 '22 09:09 hmgxr128