BlendArMocap

BlendArMocap copied to clipboard

I recorded a video for calibration purposes, which included various extreme mouth and eyebrow positions, as well as reading some text to see how convincing the mouth animation would be when synced to speech. To get the mouth movements in sync with the speech audio, it was important that the mocap animation and source video had the same number of frames (both the source video and Blender settings are at 30 fps).

That source video was 3841 frames, but after the detection process, the resulting mocap animation only lasted for 2118 frames. I repeated the detection process several times, and each time the resulting animation was 2118 frames, so it would not seem to be a hardware issue. I was able to correct the syncing by scaling the mocap animation by a factor of 1.81 (3841 / 2118)

To see if this was regular, I recorded a second video of 1303 frames. This time the detected mocap animation lasted for 722 frames, needing a scaling of 1.80 to sync it. So the scaling factor is close - at least down to the first decimal point.

Whats the video file format and what are the video dimensions? Will try to reproduce that soon

Ah yes, now I notice that the choices in prefs do not include my video. For some reason, Cheese recorded my webcam at 640 X 480 in webm. I'll look for another app to record the video.

Failing that, I have a decent Canon camera, which can take 1240 X 720 in mjpeg.

I tried another 640 X 480 video using an H264 codec in an mp4 container, but got an error.

1. Is there a preferred codec and container?
2. 720 X 640 seems unusual - I can trim the video if necessary but are those dimensions important?

Using Avidemux, I edited a video to be the preferred 720 X 640 dimension. The codec is ffmpeg ffv1 and the container matroska. This was just a short trial video (115 frames) but the mocap came to just 70 frames.

So I'm guessing the video dimensions is not the issue.

I have uploaded this video as a zip file - I hope that's useful.

trial4_edit.zip

Hi,

Does this issue is in working?

I have my video clip flipped in the highest dimension (horizontal flip if in landscape). the clip is recorded with a smartphone and the properties are:

('codec_name', 'h264') ('codec_long_name', 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10') ('profile', 'High') ('codec_time_base', '51149/3070000') ('width', 1920) ('height', 1080) ('r_frame_rate', '30/1') ('avg_frame_rate', '1535000/51149') ('time_base', '1/90000') ('start_pts', 0) ('start_time', '0.000000') ('duration_ts', 3682727) ('duration', '40.919189') ('bit_rate', '13872223') ('bits_per_raw_sample', '8') ('nb_frames', '1228')

By now, I think this issue is a opencv issue and I cannot do a lot about that. .mkv files cause this behaviour - .mp4 and .mov files do not cause this issue. Therefore I'll probably only allow to import .mp4 and .mov in the future. It's still possible that problems arise and cause this issue if the metadata of those files is wrong but that should be unusual.

Flipping the animation is intended and not a bug.