TMElyralab
does not work on new video
I Provided a new video with 25 fps configuration when I tried to inference it threw error
ading images... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 29.30it/s] get key_landmark and face bounding boxes with the bbox_shift: 5 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00, 2.70s/it] bbox_shift parameter adjustment************** Total frame:「1」 Manually adjust range : [ -47~47 ] , the current value: 5
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2.33it/s]
Inferring using: data/audio/openai-fm-coral-sympathetic.mp3
start inference
video in 25 FPS, audio idx in 50FPS
processing audio:data/audio/openai-fm-coral-sympathetic.mp3 costs 552.3748397827148ms
914
0%|▌ | 1/229 [00:01<05:33, 1.46s/it]Exception in thread Thread-2 (process_frames):
Traceback (most recent call last):
File "/root/anaconda3/envs/mustalker/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/root/anaconda3/envs/mustalker/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/ubuntu/musetalker/MuseTalk/scripts/realtime_inference.py", line 215, in process_frames
combine_frame = get_image_blending(ori_frame,res_frame,bbox,mask,mask_crop_box)
File "/home/ubuntu/musetalker/MuseTalk/musetalk/utils/blending.py", line 95, in get_image_blending
mask_image = cv2.cvtColor(mask_array,cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<1>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = cv::impl::
Invalid number of channels in input image: 'VScn::contains(scn)' where 'scn' is 1
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 229/229 [00:31<00:00, 7.18it/s]
Total process time of 914 frames including saving images = 31.896197080612183s
ffmpeg -y -v warning -r 25 -f image2 -i ./results/avatars/avator_1/tmp/%08d.png -vcodec libx264 -vf format=rgb24,scale=out_color_matrix=bt709,format=yuv420p -crf 18 ./results/avatars/avator_1/temp.mp4
[image2 @ 0x575e16c902c0] Could find no file with path './results/avatars/avator_1/tmp/%08d.png' and index in the range 0-4
[in#0 @ 0x575e16c90000] Error opening input: No such file or directory
Error opening input file ./results/avatars/avator_1/tmp/%08d.png.
Error opening input files: No such file or directory
ffmpeg -y -v warning -i data/audio/openai-fm-coral-sympathetic.mp3 -i ./results/avatars/avator_1/temp.mp4 ./results/avatars/avator_1/vid_output/audio_0.mp4
[mp3 @ 0x580a70a1b240] Estimating duration from bitrate, this may be inaccurate
[in#1 @ 0x580a70a1dcc0] Error opening input: No such file or directory
Error opening input file ./results/avatars/avator_1/temp.mp4.
Error opening input files: No such file or directory
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /root/anaconda3/envs/mustalker/lib/python3.10/runpy.py:196 in _run_module_as_main │
│ │
│ 193 │ main_globals = sys.modules["main"].dict │
│ 194 │ if alter_argv: │
│ 195 │ │ sys.argv[0] = mod_spec.origin │
│ ❱ 196 │ return _run_code(code, main_globals, None, │
│ 197 │ │ │ │ │ "main", mod_spec) │
│ 198 │
│ 199 def run_module(mod_name, init_globals=None, │
│ │
│ /root/anaconda3/envs/mustalker/lib/python3.10/runpy.py:86 in _run_code │
│ │
│ 83 │ │ │ │ │ loader = loader, │
│ 84 │ │ │ │ │ package = pkg_name, │
│ 85 │ │ │ │ │ spec = mod_spec) │
│ ❱ 86 │ exec(code, run_globals) │
│ 87 │ return run_globals │
│ 88 │
│ 89 def _run_module_code(code, init_globals=None, │
│ │
│ /home/ubuntu/musetalker/MuseTalk/scripts/realtime_inference.py:332 in
ading images... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 29.30it/s] get key_landmark and face bounding boxes with the bbox_shift: 5 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00, 2.70s/it] bbox_shift parameter adjustment************** Total frame:「1」 Manually adjust range : [ -47~47 ] , the current value: 5
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2.33it/s] Inferring using: data/audio/openai-fm-coral-sympathetic.mp3 start inference video in 25 FPS, audio idx in 50FPS processing audio:data/audio/openai-fm-coral-sympathetic.mp3 costs 552.3748397827148ms 914 0%|▌ | 1/229 [00:01<05:33, 1.46s/it]Exception in thread Thread-2 (process_frames): Traceback (most recent call last): File "/root/anaconda3/envs/mustalker/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/root/anaconda3/envs/mustalker/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/ubuntu/musetalker/MuseTalk/scripts/realtime_inference.py", line 215, in process_frames combine_frame = get_image_blending(ori_frame,res_frame,bbox,mask,mask_crop_box) File "/home/ubuntu/musetalker/MuseTalk/musetalk/utils/blending.py", line 95, in get_image_blending mask_image = cv2.cvtColor(mask_array,cv2.COLOR_BGR2GRAY) cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<1>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = cv::impl::::NONE; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
Invalid number of channels in input image: 'VScn::contains(scn)' where 'scn' is 1
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 229/229 [00:31<00:00, 7.18it/s] Total process time of 914 frames including saving images = 31.896197080612183s ffmpeg -y -v warning -r 25 -f image2 -i ./results/avatars/avator_1/tmp/%08d.png -vcodec libx264 -vf format=rgb24,scale=out_color_matrix=bt709,format=yuv420p -crf 18 ./results/avatars/avator_1/temp.mp4 [image2 @ 0x575e16c902c0] Could find no file with path './results/avatars/avator_1/tmp/%08d.png' and index in the range 0-4 [in#0 @ 0x575e16c90000] Error opening input: No such file or directory Error opening input file ./results/avatars/avator_1/tmp/%08d.png. Error opening input files: No such file or directory ffmpeg -y -v warning -i data/audio/openai-fm-coral-sympathetic.mp3 -i ./results/avatars/avator_1/temp.mp4 ./results/avatars/avator_1/vid_output/audio_0.mp4 [mp3 @ 0x580a70a1b240] Estimating duration from bitrate, this may be inaccurate [in#1 @ 0x580a70a1dcc0] Error opening input: No such file or directory Error opening input file ./results/avatars/avator_1/temp.mp4. Error opening input files: No such file or directory ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /root/anaconda3/envs/mustalker/lib/python3.10/runpy.py:196 in _run_module_as_main │ │ │ │ 193 │ main_globals = sys.modules["main"].dict │ │ 194 │ if alter_argv: │ │ 195 │ │ sys.argv[0] = mod_spec.origin │ │ ❱ 196 │ return _run_code(code, main_globals, None, │ │ 197 │ │ │ │ │ "main", mod_spec) │ │ 198 │ │ 199 def run_module(mod_name, init_globals=None, │ │ │ │ /root/anaconda3/envs/mustalker/lib/python3.10/runpy.py:86 in _run_code │ │ │ │ 83 │ │ │ │ │ loader = loader, │ │ 84 │ │ │ │ │ package = pkg_name, │ │ 85 │ │ │ │ │ spec = mod_spec) │ │ ❱ 86 │ exec(code, run_globals) │ │ 87 │ return run_globals │ │ 88 │ │ 89 def _run_module_code(code, init_globals=None, │ │ │ │ /home/ubuntu/musetalker/MuseTalk/scripts/realtime_inference.py:332 in │ │ │ │ 329 │ │ audio_clips = inference_config[avatar_id]["audio_clips"] │ │ 330 │ │ for audio_num, audio_path in audio_clips.items(): │ │ 331 │ │ │ print("Inferring using:",audio_path) │ │ ❱ 332 │ │ │ avatar.inference(audio_path, │ │ 333 │ │ │ │ │ │ │ audio_num, │ │ 334 │ │ │ │ │ │ │ args.fps, │ │ 335 │ │ │ │ │ │ │ args.skip_save_images) │ │ │ │ /home/ubuntu/musetalker/MuseTalk/scripts/realtime_inference.py:283 in inference │ │ │ │ 280 │ │ │ print(cmd_combine_audio) │ │ 281 │ │ │ os.system(cmd_combine_audio) │ │ 282 │ │ │ │ │ ❱ 283 │ │ │ os.remove(f"{self.avatar_path}/temp.mp4") │ │ 284 │ │ │ shutil.rmtree(f"{self.avatar_path}/tmp") │ │ 285 │ │ │ print(f"result is save to {output_vid}") │ │ 286 │ │ print("\n") │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ FileNotFoundError: [Errno 2] No such file or directory: './results/avatars/avator_1/temp.mp4' (mustalker) root@t1-le-45-gra7:/home/ubuntu/musetalker/MuseTalk#
check https://github.com/TMElyralab/MuseTalk/issues/220#issuecomment-2467139770 it helps me so I hope it help you too
