[Bug] Pose3DInferencer (IndexError: list index out of range)

[Va6lue]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] The bug has not been fixed in the latest version(https://github.com/open-mmlab/mmpose).

Environment

OrderedDict([('sys.platform', 'win32'), ('Python', '3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)]'), ('CUDA available', True), ('MUSA available', False), ('numpy_random_seed', 2147483648), ('GPU 0', 'NVIDIA GeForce RTX 4090'), ('CUDA_HOME', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1'), ('NVCC', 'Cuda compilation tools, release 12.1, V12.1.66'), ('MSVC', 'Microsoft (R) C/C++ Optimizing Compiler Version 19.41.34120 for x64'), ('GCC', 'n/a'), ('PyTorch', '2.3.1+cu121'), ('PyTorch compiling details', 'PyTorch built with:\n - C++ Version: 201703\n - MSVC 192930154\n - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications\n - Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361)\n - OpenMP 2019\n - LAPACK is enabled (usually provided by MKL)\n - CPU capability usage: AVX512\n - CUDA Runtime 12.1\n - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n - CuDNN 8.9.7 (built against CUDA 12.2)\n - Magma 2.5.4\n - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.7, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /Zc:__cplusplus /bigobj /FS /utf-8 -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.1, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, \n'), ('TorchVision', '0.18.1+cu121'), ('OpenCV', '4.10.0'), ('MMEngine', '0.10.4'), ('MMPose', '1.3.2+')])

pip list: addict==2.4.0 aliyun-python-sdk-core==2.15.1 aliyun-python-sdk-kms==2.16.4 certifi==2024.7.4 cffi==1.17.0 charset-normalizer==3.3.2 chumpy==0.70 click==8.1.7 colorama==0.4.6 contourpy==1.2.1 crcmod==1.7 cryptography==43.0.0 cycler==0.12.1 Cython==3.0.11 filelock==3.14.0 fonttools==4.53.1 fsspec==2024.2.0 idna==3.8 importlib_metadata==8.4.0 intel-openmp==2021.4.0 Jinja2==3.1.3 jmespath==0.10.0 json-tricks==3.17.3 kiwisolver==1.4.5 Markdown==3.7 markdown-it-py==3.0.0 MarkupSafe==2.1.5 matplotlib==3.9.2 mdurl==0.1.2 mkl==2021.4.0 mmcv==2.2.0 mmdet==3.3.0 mmengine==0.10.4 mmpose==1.3.2 model-index==0.1.11 mpmath==1.3.0 munkres==1.1.4 networkx==3.2.1 numpy==1.26.3 opencv-python==4.10.0.84 opendatalab==0.0.10 openmim==0.3.9 openxlab==0.1.1 ordered-set==4.1.0 oss2==2.17.0 packaging==24.1 pandas==2.2.2 parse==1.20.2 pillow==10.2.0 platformdirs==4.2.2 pycocotools==2.0.8 pycparser==2.22 pycryptodome==3.20.0 Pygments==2.18.0 pyparsing==3.1.4 python-dateutil==2.9.0.post0 pytz==2023.4 pywin32==306 PyYAML==6.0.2 regex==2024.7.24 requests==2.28.2 rich==13.4.2 scipy==1.14.1 shapely==2.0.6 six==1.16.0 sympy==1.12 tabulate==0.9.0 tbb==2021.11.0 termcolor==2.4.0 terminaltables==3.1.10 tomli==2.0.1 torch==2.3.1+cu121 torchaudio==2.3.1+cu121 torchvision==0.18.1+cu121 tqdm==4.66.5 typing_extensions==4.9.0 tzdata==2024.1 urllib3==1.26.19 xtcocotools==1.14.3 yapf==0.40.2 zipp==3.20.1

Reproduces the problem - code sample

from mmpose.apis import MMPoseInferencer
from tqdm import tqdm
from pathlib import Path

def test_bug(inferencer, p):
    result_generator = inferencer(str(p), show=False)
    for result in result_generator:
        pass

if __name__ == '__main__':
    my_dataset_folder = Path("C:/dataset")  # put different videos in this directory
    all_mp4_paths = sorted(my_dataset_folder.glob('**/*.mp4'))
    
    inferencer = MMPoseInferencer(pose3d='human3d')
    pbar = tqdm(range(len(all_mp4_paths)), desc='Yield .npy files', unit='video')
    for video_path in all_mp4_paths:
        test_bug(inferencer, video_path)

Reproduces the problem - command or script

python XXX.py

Reproduces the problem - error message

IndexError: list index out of range File "C:\MyResearch\stroke_classification\venv_prepare_train\Lib\site-packages\mmpose\apis\inferencers\pose3d_inferencer.py", line 325, in forward pose_lift_res.track_id = pose_est_results_converted[idx].get(

Additional information

1. It should be like using the 2D inferencer, there isn't any error there.
2. Some .mp4 files are save to pass the 3D inferencer code, but some aren't.
3. Although it works on creating a MMPoseInferencer object per video, that is not a efficient way.