About Dance Editing

[lzyplayer]

Thanks for sharing such great work! I saw in the paper that EDGE is capable of Motion Editing. I want to know how to use them in the demo. Provided with first and last pose of motion.

[lzyplayer]

I found the mode "inpaint" in diffusion.py. It might be the point.

[lvZic]

I found the mode "inpaint" in diffusion.py. It might be the point.

have u used the "inpaint" mode successfully?

[KevinGoodman]

I try to use the inpaint_loop function in diffusion.py, and the constraint['value'] is just the GT motion from AIST++, I use the following mask to try inpainting/in-betweening/joint-wise constraint. However, the result I got is far from satisfaction, as the constraint seems not to be working(in inpainting when the sample rate is low, I try 10%/30%/50%/70%, 30%+ gets better), and the motions generated are not so smooth

for inpainting:

mask = constraint["mask"].to(device)  # [b, l, 1]
mask = torch.zeros_like(x)
inpaint
l = x.shape[1]
sample_rate = .7
rand_idx = torch.randperm(l)[:int(l * sample_rate)]
mask[:, rand_idx] = 1.

for in-betweening:

mask = torch.zeros_like(x)
mask[:, 0] = 1.
mask[:, -1] = 1.

for joint-wise constraint:

mask = torch.ones_like(x)
down_joint_mask = [
    1, # Pelvis
    1, 1,   # LeftHip, RightHip
    0,  # Spine1
    1, 1,   # LeftKnee, RightKnee
    0,  # Spine2
    1, 1,   # LeftAnkle, RightAnkle
    0,  # Spine3
    1, 1,    # LeftFoot, RightFoot
    0, # Neck
    0, 0,   # LeftCollar, RightCollar
    0, # Head
    0, 0,   # LeftShoulder, RightShoulder
    0, 0,   # LeftElbow, RightElbow
    0, 0,   # LeftWrist, RightWrist
    0, 0,   # LeftHand, RightHand
]
mask[:, :, 7:] *= (torch.tensor(down_joint_mask, device=x.device).repeat_interleave(6).unsqueeze(0).unsqueeze(0))
# mask[:, :, 7:] *= (1 - torch.tensor(down_joint_mask, device=x.device).repeat_interleave(6).unsqueeze(0).unsqueeze(0))   # up joint mask

Any advice would be greatly appreciated