ffcv
ffcv copied to clipboard
Make RGBImageField support image file
When I try to make my own dataset, I find the DatasetWriter and RGBImageField does not support directly converting the image file. It would be great if we can just pass the image path to the Field and read the jpeg, png file's metadata and byte. This method would be faster than read image to tensor and encode tensor to byte.
Hi @zero0kiriyu ! In general, this should be really simple to do---FFCV can convert any indexable object into an FFCV dataset! See this guide for an example: https://docs.ffcv.io/writing_datasets.html.
Something like ffcv.fields.BytesField
could be what you are looking for if you really want to avoid ever reading the images into tensor format, but I think this would be strictly slower (at loading time) than using the standard method.
@andrewilyas Probably my question is related to this: I want to use FFCV on my tensor dataset, not ImageFolder, because I need to train SimCLR and my dataset has pairs of augmentations in the tensor type. How is that possible?