GeneFacePlusPlus
GeneFacePlusPlus copied to clipboard
yerfor
Data Preprocessing Error
I am trying to preprocess the video https://drive.google.com/file/d/1e7VejTUh9fEg-DHEdestvHZJSx1FklBU/view?usp=sharing , but it is giving the following errors.
video:811654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame=11835 fps=2369 q=1.0 Lsize=N/A time=00:07:53.36 bitrate=N/A speed=94.7x
todo videos number: 1
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1714576703.928276 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
| Extracting Segmaps && Saving...
generating segment images in multi-processes...: 0%| | 0/11835 [00:00<?, ?it/s]I0000 00:00:1714576704.382687 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.383241 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.384124 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.386262 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.387167 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.387454 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.388938 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.388938 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.390370 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.393299 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.394696 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.397276 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.403220 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.405642 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.434605 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.454487 289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
generating segment images in multi-processes...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 11835/11835 [06:24<00:00, 30.76it/s]
| Extracted Segmaps Done.
| Extracting background...
combining backgrounds...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 515/515 [04:24<00:00, 1.95it/s]
<class 'ValueError'> Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required by NearestNeighbors.
Traceback (most recent call last):
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 361, in extract_segment_job
bg_img = extract_background(img_lst, segmap_mask_lst, method=background_method, device=device, mix_bg=mix_bg)
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 140, in extract_background
distances, indices = nbrs.kneighbors(bg_xys) # 对non-bg img，用KNN找最近的bg pixel
File "/opt/conda/envs/geneface/lib/python3.9/site-packages/sklearn/neighbors/_base.py", line 826, in kneighbors
X = self._validate_data(X, accept_sparse="csr", reset=False, order="C")
File "/opt/conda/envs/geneface/lib/python3.9/site-packages/sklearn/base.py", line 633, in _validate_data
out = check_array(X, input_name="X", **check_params)
File "/opt/conda/envs/geneface/lib/python3.9/site-packages/sklearn/utils/validation.py", line 1072, in check_array
raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required by NearestNeighbors.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 491, in <module>
extract_job(*fn_args[0])
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 384, in extract_segment_job
traceback.print_exc(e)
File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 163, in print_exc
print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 103, in print_exception
for line in TracebackException(
File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 517, in __init__
self.stack = StackSummary.extract(
File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 340, in extract
if limit >= 0:
TypeError: '>=' not supported between instances of 'ValueError' and 'int'
todo videos number: 1
Root 0: extracing MP-based landmark2d: 0%| | 0/1 [00:00<?, ?it/s]WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1714577393.724594 292523 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577393.725103 292523 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
I0000 00:00:1714577393.774896 292523 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577393.775419 292523 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
Warning: failed detect ldm in idx=0, use previous frame results.
Traceback (most recent call last):
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_lm2d.py", line 75, in extract_landmark_job
img_lm478, vid_lm478 = face_landmarker.extract_lm478_from_video_name(video_name)
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/mp_feature_extractors/face_landmarker.py", line 80, in extract_lm478_from_video_name
img_lm478, vid_lm478 = self.extract_lm478_from_frames(frames, fps, anti_smooth_factor)
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/mp_feature_extractors/face_landmarker.py", line 102, in extract_lm478_from_frames
img_face_landmarks = np.array([[l.x, l.y, l.z] for l in img_ldm_i])
UnboundLocalError: local variable 'img_ldm_i' referenced before assignment
finished 1 / 1 = 1.0000, failed 1 / 1 = 1.0000
Root 0: extracing MP-based landmark2d: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:09<00:00, 9.64s/it]
args Namespace(vid_dir='data/raw/videos/bipolar_2.mp4', ds_name='nerf', seed=0, process_id=0, total_process=1, id_mode='global', keypoint_mode='mediapipe', large_yaw_threshold=9999999.9, debug=True, reset=True, load_names=False)
['data/raw/videos/bipolar_2.mp4']
process 0: fitting 3dmm ...: 0%| | 0/1 [00:00<?, ?it/s]lms_2d file not found, try to extract it from video... data/processed/videos/bipolar_2/lms_2d.npy
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1714577403.960238 292647 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577403.960656 292647 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
I0000 00:00:1714577404.004503 292647 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577404.004936 292647 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
Warning: failed detect ldm in idx=0, use previous frame results.
local variable 'img_ldm_i' referenced before assignment
finished 1 / 1 = 1.0000, failed 1 / 1 = 1.0000
process 0: fitting 3dmm ...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:06<00:00, 6.76s/it]
all failed image names: ['data/raw/videos/bipolar_2.mp4']
All finished!
Traceback (most recent call last):
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/runs/binarizer_nerf.py", line 338, in <module>
binarizer.parse(video_id)
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/runs/binarizer_nerf.py", line 317, in parse
ret = load_processed_data(processed_dir)
File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/runs/binarizer_nerf.py", line 200, in load_processed_data
assert os.path.exists(background_img_name)
AssertionError
I met the similar issue, have you solved it? The error is because bg.jpg is not present in the directory GeneFacePlusPlus/data/processed/videos/${VIDEO_ID}/bg.jpg once again try the complete data preprocessing step it worked for me
I met the similar issue, have you solved it? The error is because bg.jpg is not present in the directory GeneFacePlusPlus/data/processed/videos/${VIDEO_ID}/bg.jpg once again try the complete data preprocessing step it worked for me
thanks for your reply, so how to extract bg.jpg
重新跑一下命令
python data_gen/utils/process_video/extract_segment_imgs.py --ds_name=nerf --vid_dir=data/raw/videos/${VIDEO_ID}.mp4 --force_single_process
hoverZ
You made my day. But can you elaborate about the problem?
大概率执行 python data_gen/utils/process_video/extract_segment_imgs.py --ds_name=nerf --vid_dir=data/raw/videos/${VIDEO_ID}.mp4 是程序卡住，没有执行成功，导致文件没有产生。根本原因可能是代码在处理多线程时，没处理好吧
python data_gen/utils/process_video/extract_segment_imgs.py --ds_name=nerf --vid_dir=data/raw/videos/${VIDEO_ID}.mp4 --force_single_process
I tried this 3-4 times but still I am getting error like
Warning: failed detect ldm in idx=0, use previous frame results. Traceback (most recent call last): File "/home/arulmozg/GeneFacePlusPlus-LipSync/data_gen/utils/process_video/extract_lm2d.py", line 75, in extract_landmark_job img_lm478, vid_lm478 = face_landmarker.extract_lm478_from_video_name(video_name) File "/home/arulmozg/GeneFacePlusPlus-LipSync/data_gen/utils/mp_feature_extractors/face_landmarker.py", line 80, in extract_lm478_from_video_name img_lm478, vid_lm478 = self.extract_lm478_from_frames(frames, fps, anti_smooth_factor) File "/home/arulmozg/GeneFacePlusPlus-LipSync/data_gen/utils/mp_feature_extractors/face_landmarker.py", line 102, in extract_lm478_from_frames img_face_landmarks = np.array([[l.x, l.y, l.z] for l in img_ldm_i]) UnboundLocalError: local variable 'img_ldm_i' referenced before assignment finished 1 / 1 = 1.0000, failed 1 / 1 = 1.0000
is that what the real issue? I think is it due to the background color/ video resolution? Any inputs on this?
