ffcv icon indicating copy to clipboard operation
ffcv copied to clipboard

Adapt to different image sizes

Open Breeze-Zero opened this issue 1 year ago • 5 comments

My task requires that the images do not scale and crop uniformly, so I can only batch size 1 to import different sizes of data. Can FFCV writer achieve this?

Breeze-Zero avatar Jul 12 '23 07:07 Breeze-Zero

Have you used the RandomResizedCrop() function, or ResizedCropRGBImageDecoder() class?

tavisshore avatar Jul 12 '23 09:07 tavisshore

Have you used the RandomResizedCrop() function, or ResizedCropRGBImageDecoder() class?

No, Just reading the data, the data itself is not consistent and I don't want to resize it.

Breeze-Zero avatar Jul 12 '23 09:07 Breeze-Zero

What are you actually trying to do

tavisshore avatar Jul 12 '23 09:07 tavisshore

What are you actually trying to do

Here's the size range of my data: {(448, 132, 10, 9, 12), (448, 132, 10, 10, 12), (512, 204, 10, 11, 12), (512, 204, 10, 12, 12), (512, 204, 10, 13, 12). (512, 246, 10, 11, 12), (512, 204, 10, 9, 12), (512, 204, 10, 10, 12), (512, 246, 10, 9, 12), (512, 162, 10, 9, 12), (512, 246, 10, 10, 12), (512, 162, 10, 10, 12), (512, 246, 10, 12, 12)}, some dimensions are not consistent. I can't use cropping or scaling operations due to the peculiarity of the data, so I have to read it into the dataloader originally. The model is trained in a batch size. Whether the Writer of FFCV can support this kind of data without changing the size of the data?

Breeze-Zero avatar Jul 12 '23 09:07 Breeze-Zero

As far as I'm aware the writer can create the beton file fine with varying sizes, but you won't be able to read it effectively without having to crop / resize to make all tensors within a batch the same shape.

tavisshore avatar Jul 12 '23 09:07 tavisshore