AniPortrait icon indicating copy to clipboard operation
AniPortrait copied to clipboard

音频时长2分钟,生成视频的时候会报错IndexError index 3450 is out of bounds for axis 0 with size 3450

Open ghjuh opened this issue 1 year ago • 1 comments

ile "/root/autodl-tmp/sandu1/ramouswu/AniPortrait/scripts/audio2vid.py", line 200, in main projected_vertices = project_points(pred, face_result['trans_mat'], pose_seq, [height, width]) File "/root/autodl-tmp/sandu1/ramouswu/AniPortrait/src/utils/pose_util.py", line 38, in project_points transformed_points = points_3d_homogeneous @ (transformation_matrix @ euler_and_translation_to_matrix(pose_vectors[i][:3], pose_vectors[i][3:])).T @ P IndexError: index 3450 is out of bounds for axis 0 with size 3450

ghjuh avatar Nov 08 '24 08:11 ghjuh

One possible reason is that

sample["seq_len"] % sr * chunk_duration == 0

which results in

seq_len_list = [chunk_duration*fps] * (len(audio_chunks) - 1) + [sample['seq_len'] % (chunk_duration*fps)] # 30 fps 

being shorter than it should be. For example, [530, 530, 530, 0], where 0 should be 530 My fix is the following:

                audio_chunks = list(sample['audio_feature'].split(chunk_size, dim=1))
                if len(audio_chunks) == 1:
                    seq_len_list = [sample['seq_len']]
                elif sample["seq_len"] % (chunk_duration * fps) != 0:
                    seq_len_list = [chunk_duration*fps] * (len(audio_chunks) - 1) + [sample['seq_len'] % (chunk_duration*fps)] # 30 fps 
                    audio_chunks[-2] = torch.cat((audio_chunks[-2], audio_chunks[-1]), dim=1)
                    seq_len_list[-2] = seq_len_list[-2] + seq_len_list[-1]
                    del audio_chunks[-1]
                    del seq_len_list[-1]
                else:
                    seq_len_list = [chunk_duration*fps] * len(audio_chunks)

ladvu avatar Dec 23 '24 16:12 ladvu