Error in running custom target video

[AustinZzx]

Got following error:

Traceback (most recent call last):
  File "/home/zexia/GeneFace/data_util/process.py", line 437, in <module>
    extract_background(processed_dir, ori_imgs_dir)
  File "/home/zexia/GeneFace/data_util/process.py", line 124, in extract_background
    distances, indices = nbrs.kneighbors(bg_xys)
  File "/home/zexia/miniconda3/envs/geneface/lib/python3.9/site-packages/sklearn/neighbors/_base.py", line 804, in kneighbors
    X = self._validate_data(X, accept_sparse="csr", reset=False, order="C")
  File "/home/zexia/miniconda3/envs/geneface/lib/python3.9/site-packages/sklearn/base.py", line 605, in _validate_data
    out = check_array(X, input_name="X", **check_params)
  File "/home/zexia/miniconda3/envs/geneface/lib/python3.9/site-packages/sklearn/utils/validation.py", line 967, in check_array
    raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required by NearestNeighbors.
[ WARN:[email protected]] global loadsave.cpp:248 findDecoder imread_('data/processed/videos/bag/bc.jpg'): can't open/read file: check file path/integrity
[INFO] ===== extract head images for data/processed/videos/bag =====
  0%|                                                                                      | 0/2109 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/zexia/GeneFace/data_util/process.py", line 441, in <module>
    extract_head(processed_dir)
  File "/home/zexia/GeneFace/data_util/process.py", line 148, in extract_head
    img[~head_part] = bg_img[~head_part]
TypeError: 'NoneType' object is not subscriptable
[INFO] ===== save transforms =====
Traceback (most recent call last):
  File "/home/zexia/GeneFace/data_util/process.py", line 454, in <module>
    save_transforms(processed_dir, ori_imgs_dir)
  File "/home/zexia/GeneFace/data_util/process.py", line 302, in save_transforms
    params_dict = torch.load(os.path.join(base_dir, 'track_params.pt'))
  File "/home/zexia/miniconda3/envs/geneface/lib/python3.9/site-packages/torch/serialization.py", line 986, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/zexia/miniconda3/envs/geneface/lib/python3.9/site-packages/torch/serialization.py", line 435, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/zexia/miniconda3/envs/geneface/lib/python3.9/site-packages/torch/serialization.py", line 416, in __init__
    super().__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'data/processed/videos/bag/track_params.pt'
Loading the Wav2Vec2 Processor...
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Loading the HuBERT Model...
Some weights of HubertModel were not initialized from the model checkpoint at facebook/hubert-large-ls960-ft and are newly initialized: ['hubert.encoder.pos_conv_embed.conv.parametrizations.weight.original1', 'hubert.encoder.pos_conv_embed.conv.parametrizations.weight.original0']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
2023-11-12 16:45:17.180545: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-11-12 16:45:17.266603: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-11-12 16:45:17.284870: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-11-12 16:45:17.595099: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/usr/local/cuda-11.8/lib64:/usr/local/cuda-11.8/lib64
2023-11-12 16:45:17.595176: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/usr/local/cuda-11.8/lib64:/usr/local/cuda-11.8/lib64
2023-11-12 16:45:17.595191: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Hubert extracted at data/processed/videos/bag/aud_hubert.npy
Mel and F0 extracted at data/processed/videos/bag/aud_mel_f0.npy
loading the model from deep_3drecon/checkpoints/facerecon/epoch_20.pth
loading video ...
extracting 2D facial landmarks ...:  19%|███████▌                                | 429/2286 [00:52<03:00, 10.28it/s]WARNING: Caught errors when fa.get_landmarks, maybe No face detected at frame 2286 in data/raw/videos/bag.mp4!
extracting 2D facial landmarks ...:  19%|███████▌                                | 430/2286 [00:52<03:45,  8.22it/s]
Traceback (most recent call last):
  File "/home/zexia/GeneFace/data_gen/nerf/extract_3dmm.py", line 56, in process_video
    lm68 = fa.get_landmarks_from_image(frames[i])[0] # 识别图片中的人脸，获得角点, shape=[68,2]
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zexia/GeneFace/data_gen/nerf/extract_3dmm.py", line 112, in <module>
    process_video(video_fname, out_fname, skip_tmp=False)
  File "/home/zexia/GeneFace/data_gen/nerf/extract_3dmm.py", line 59, in process_video
    raise ValueError("")
ValueError
| Unknow hparams:  []
Traceback (most recent call last):
  File "/home/zexia/GeneFace/data_gen/nerf/binarizer.py", line 14, in <module>
    set_hparams()
  File "/home/zexia/GeneFace/utils/commons/hparams.py", line 46, in set_hparams
    assert os.path.exists(args.config)
AssertionError

[yerfor]

Hi, it seems that in the 430th frame of the training video, the landmark detector has not find a face. Please check the 430th frame of the video, and consider deleting the bad frames that don't have a detected face.