Janusz Lisiecki

Hi @zhanghang-cv, For now, DALI creates a libaviutil context for each video in the dataset - see https://github.com/NVIDIA/DALI/issues/2220 for more details. So in your case, if you have 1 million...

Hi @zhanghang-cv, How exactly do you measure memory consumption? Can the increase come from the fact that [OS is caching the data](https://www.baeldung.com/linux/empty-buffer-cache#:~:text=The%20cache%20in%20Linux%20is,the%20requested%20blocks%20of%20data.) in RAM when accessing it from the drive?

DALI doesn't use RAM to store decoded video for the GPU video decoder, so it doesn't seem to be the reason.

@Freed-Wu, It looks like a DALI bug but I'm unable to create the test dataset the way you did it (the script you shared lacks `get_dataloader` definition). Can you create...

Hi @Freed-Wu, I have managed to reproduce the problem. In your case the `fn.readers.caffe` returns empty image/s and DALI cannot handle the transfer of an empty tensor to the GPU....

You can cast it to int8 and the use DALI [`reinterpret` operator](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/operations/nvidia.dali.fn.reinterpret.html#nvidia.dali.fn.reinterpret) to cast it back to int32.

By cast, I mean different interpretations of the underlying data, like [numpy view](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.view.html).

Can you share the updated data set?

As I understand, the width and height provided in each entry are hints, unless you set `encoded=False` to indicate that raw images are stored, which doesn't require any decoding. Otherwise,...

I was talking about LMDB schema mentioned in https://github.com/NVIDIA/DALI/issues/4254#issuecomment-1247064906. Regarding the `fn.reinterpret` operator you can set the layout ("CHW" or "HWC") and shape, in this case, I would use `rel;_shape=[1,...