speech2gesture
speech2gesture copied to clipboard
Misalignment of audio and frames
According to the file common/consts.py
, we know that
SR = 16000
AUDIO_SHAPE = 67267
FPS = 15
FRAMES_PER_SAMPLE = 64
From the first three constants, we can compute num_frames = AUDIO_SHAPE / SR * FPS = 67267 / 16000 * 15 = 63.06281249999999
which is about one whole frame less than FRAMES_PER_SAMPLE
.
We have encountered this problem when we were trying to test the model on a longer audio sequence, for which the misalignment is magnified.