Creation of FFFR CPU backend hangs after importing PyAV

[anibali]

Attempting to create an FFFR CPU backend after importing PyAV (import av) will result in a freeze. PyAV interacts with the ffmpeg libraries which FFFR also uses, so this is likely triggering the issue in some way. Note that when using FFFR for GPU decoding, there is no issue.

Steps to reproduce

import torch
from tvl_backends.fffr import FffrBackendFactory

# Removing this import will prevent the freeze.
import av

# The video file doesn't matter
video_file = '/home/aiden/Projects/PyTorch/tvl/data/board_game-h264.mkv'

# This will hang indefinitely
backend = FffrBackendFactory().create(video_file, 'cpu', torch.float32)

Tracing the issue

• From the TVL side, the call to Ffr::Stream::getStream in TvFFFrameReader.cpp never returns.
• Digging deeper into FFFrameReader, the freeze occurs in Stream::decodeNextBlock, and more specifically it's the call to avcodec_send_packet which never returns (https://github.com/Sibras/FFFrameReader/blob/b35e7838ced77c3f62c483e4fccf38e8dd881e22/source/FFFRStream.cpp#L798).