ffcv
ffcv copied to clipboard
Problem while unpacking encoded JSONField
Hello,
I am using FFCV for an object detection task. When writing the .beton file, I am using RGBImageField for the image and JSONField for writing the dict of bounding boxes.
writer = DatasetWriter(path, {
'image': RGBImageField(),
'label': JSONField()
}
)
writer.from_indexed_dataset(ds)
Writing the .beton file works properly with no problems. However, when I am looping over the batches after loading the .beton I am getting an error while unpacking the label field to get the json back.
for i, batch in enumerate(train_data_loader):
x, y = batch
#this line sometimes through an error
y = JSONField.unpack(y)
The error says the following:
sep_location = np.where(b == ord(SEPARATOR))[0][0]
IndexError: index 0 is out of bounds for axis 0 with size 0
Which basically means that the SEPARATOR does not exist in the encoded array.
This happens only to few number of data points, any thoughts?