Video doesn't seem to use the context of previous/next frames

[rebroad]

I'm seeing a person's eye colour and ethnicity changing from frame to frame - a good feature might be to take into account the variables from the frames before and after to reduce this happening.

Also, it would be useful if I could tell it the ages of the people in the video. For example, I am running it against a video where everyone is the same age in the video, age 17, and yet it's giving many of the people beards when none of the people had beards - it wasn't so fashionable in 1989. It would be useful if it could factor in things such as peoples' ages, and the year (to take the fashions into account), and also perhaps the country (to take ethnicity into account, albeit risky to make assumptions).

[weizmann]

I found the fidelity weight w is related to your question from my struggle experience.

When I set w to 0, the frames are shaking but with high quality, I think the model is "imaging" some details. When I set w to 1, the frames are stable.

Cited from README: Fidelity weight w lays in [0, 1]. Generally, smaller w tends to produce a higher-quality result, while larger w yields a higher-fidelity result.

[rebroad]

I found the fidelity weight w is related to your question from my struggle experience.

When I set w to 0, the frames are shaking but with high quality, I think the model is "imaging" some details. When I set w to 1, the frames are stable.

Cited from README: Fidelity weight w lays in [0, 1]. Generally, smaller w tends to produce a higher-quality result, while larger w yields a higher-fidelity result.

I was using fidelity weight 0.7 I believe, but at these numbers it changes the identity of a person significantly, so they are no longer recognisable to people who know them.

For the video remastering, I have now switched to GFPGAN and this is producing much better results, and is also much faster per frame. I will probably switch to supporting that project with regards to improving the upscaling of old/blurry video.

[zhewangcn]

I met the similar problems, codeformer with lower w gives great face restoration results, but faces are a little bit different across frames and the video looks flickering. Using a high w make it stable, but the face looks not so perfect.

Any ideas on how to solve this inconsistency with lower w value? Curious how would GFPGAN generate stable images between frames.

[fz5400]

I have an idea: it may be possible to get a stable facial video by swapping the face with one of the frames of the video generated by codeformer

[fz5400]

I have an idea: it may be possible to get a stable facial video by swapping the face with one of the frames of the video generated by codeformer

Using some face swapping model such as deep face lab