readers.video hang on reading avi files

[cnut1648]

Hi, I am using DALI on NTU RGB D dataset, and for some reasons, my code hangs when reading the avi files, similar to @3312. In @3312 it seems that this issue is related to VFR. But I see that there is a flag skip_vfr_check, so I guess now VFR functionality is already incorporated? My code is

@pipeline_def
def get_dali_pipeline(crop_size):
    
    data_dir = "/home/ICT2000/jxu/Multimodal-Transformer/data/data/NTU RGB+D/nturgb+d_rgb"
    csv_dir = "/home/ICT2000/jxu/Multimodal-Transformer/data/datasets/NTU RGB+D"
    df = pd.read_csv(f"{csv_dir}/post_val.csv")
    files, labels = (data_dir + "/" + df["path"]).tolist(), df["label"].tolist()

    images = fn.readers.video(
        device="gpu", filenames=files, random_shuffle=False, sequence_length=8,
        name="Reader", dtype=types.UINT8, pad_last_batch=True, image_type=types.RGB,)
    images = fn.crop(images, crop=crop_size, dtype=types.FLOAT,
                     crop_pos_x=fn.random.uniform(range=(0.0, 1.0)),
                     crop_pos_y=fn.random.uniform(range=(0.0, 1.0)))
    images = fn.transpose(images, perm=[3, 0, 1, 2])

    
    return images

pipeline = get_dali_pipeline(
                    num_threads=1,
                    batch_size=4,
                    crop_size=224)
pipeline.build()
train_data = DALIGenericIterator(
    pipeline, ["data"],
   reader_name='Reader', auto_reset=True,
)


for i, data in enumerate(train_data):
  x = data[0]['data']

Please let me know if I made some mistakes in my code. Thanks!

[JanuszL]

Hi @cnut1648,

The video reader operator doesn't support variable frame rate videos. We added a heuristics that should detect this kind of case, still, the heuristic is not 100% accurate and we added an ability to turn it off - skip_vfr_check. The next experimental.readers.video operator supports vfr, still its functionality is rudimentary. You can try it out to see if that helps.