magic-animate icon indicating copy to clipboard operation
magic-animate copied to clipboard

Using multiple reference images

Open niqodea opened this issue 1 year ago • 0 comments

Hi, thank you for your work, really impressive results!

In my problem domain I have access to several reference images of the scene I would like to generate, not just one. To make the best out of this information, I was thinking of feeding to the reference appearance encoder all of them and then take the average. I tried to implement it by modifying the source code but the complex hook logic makes this change not that trivial. I had two questions for you:

  1. Do you think averaging the appearance encoding of multiple frames can generate a more precise encoding of the appearance that the overall video should have? Have you tried this in your experiments?
  2. Do you think there is an easy way to do this by modifying relatively little of the source code that I am missing?

Thank you again for your time!

niqodea avatar Dec 12 '23 13:12 niqodea