Images not paired

[bezorro]

Hi, thanks for your high quality dataset. But I find that the images in images/with_ground_truth/bunny_3Lights_rotateUp are not paired. e.g. GuideColored0593.png and HighQualityRender0593.png is not matched of the rotation angles. It seems that GuideColored is two times the angular speed of HighQualityRender. Is it a mistake or it supposed to be?

[trungtle]

Hi bezorro, this might have been a mistake where one set was generated as twice the fps of the other set. You might have to rename these frames for them to match. Unfortunately I don't have access to the render farm any more to regenerate the images.

[bezorro]

Thanks @trungtle. Sorry for the late reply. I paired the images by myself and trained the model well. Your method works. I used larger UNet or even HRNetV2 to even get results slightly better(sharper) than yours. I am now fighting for videos that are sharper and more temporally stable. Have you refine the model after that or any new ideas/advice for better results?

[trungtle]

Hi @bezorro, unfortunately I since left Google, so I no longer have access to the models.

Out of curious, what results have you seen so far?

Getting it to be temporally stable is pretty challenging. Have you tried varying the number of previous frames included in the training (in my experience less # frames tend to perform better). Perhaps, estimate the optical flow from the training data could also help stabilizing it? There is also the hair's point velocity information that you could extract from the Alembic file, that potentially could help generate additional training data for flow.