IP_LAP icon indicating copy to clipboard operation
IP_LAP copied to clipboard

lip_embedding and jaw_embedding

Open Aditya870 opened this issue 4 months ago • 0 comments

How to know N_l:N_l+T is lip_embedding and N_l+T: is jaw_embedding. As used in code below. I am using more no of landmark points. so i need to know how you are getting this information. The code is attached below:

#3. fuse embedding
output_tokens=self.fusion_transformer(ref_embedding,mel_embedding,pose_embedding)

#4.output  landmark
**lip_embedding=output_tokens[:,N_l:N_l+T,:] #(B,T,dim)
jaw_embedding=output_tokens[:,N_l+T:,:] #(B,T,dim)**
output_mouse_landmark=self.mouse_keypoint_map(lip_embedding)  ##(B,T,40*2)
output_jaw_landmark=self.jaw_keypoint_map(jaw_embedding)   ##(B,T,17*2)

Aditya870 avatar Feb 24 '24 07:02 Aditya870