speech2gesture icon indicating copy to clipboard operation
speech2gesture copied to clipboard

Misalignment of audio and frames

Open ShenhanQian opened this issue 4 years ago • 0 comments

According to the file common/consts.py, we know that

SR = 16000
AUDIO_SHAPE = 67267
FPS = 15
FRAMES_PER_SAMPLE = 64

From the first three constants, we can compute num_frames = AUDIO_SHAPE / SR * FPS = 67267 / 16000 * 15 = 63.06281249999999 which is about one whole frame less than FRAMES_PER_SAMPLE.

We have encountered this problem when we were trying to test the model on a longer audio sequence, for which the misalignment is magnified.

ShenhanQian avatar Jan 19 '21 08:01 ShenhanQian