flowframes
flowframes copied to clipboard
n00mkrad
Request: Integrate IFRNet
IFRNet: https://github.com/ltkong218/ifrnet
Even has an ncnn version: https://github.com/nihui/ifrnet-ncnn-vulkan
It seems to score much higher than even XVFI according to the papers at: https://paperswithcode.com/sota/video-frame-interpolation-on-vimeo90k (scoring second against many algorithms)
This would be a wonderful addition to flowframes!
Already did an experimental implementation, it's mediocre.
Will still make it available soon
Second this. IFRNet seems to be promising. @n00mkrad is RIFE really still the best interp method? Or do you rather mean the best real-time method?
Second this. IFRNet seems to be promising. @n00mkrad is RIFE really still the best interp method? Or do you rather mean the best real-time method?
RIFE still
@FelixKainz After more testing, IFRNet on tta mode (the -x option on the cli) seems to greatly outperform rife even with rife's spatial and temporal tta modes on.
Im very impressed with IFRNet so far. I am using the IFRNet_L_Vimeo90K model. Because it contains a larger dataset than the GoPro model.
A downside being it takes a lot longer...
IFRNet on tta mode (the -x option on the cli) seems to greatly outperform rife even with rife's spatial and temporal tta modes on.
Intersting. How much longer does it take? Did you try to find the point of deminishing returns as in how long will each take to produce a similar result? If it takes IFRNet only like two or three times as long it would still be a viable alternative for non-real-time use cases.
If you can give more details on your testing, I would appreciate to hear @n00mkrad's take on it!
I have only tested it on 2d animation. So it may be slightly biased. It seems to improve coherency quite a bit versus non tta mode. But the speed reduction is quite insane. I'd say it's probably not worth it unless your footage is a minute or less, or unless you had multiple powerful cards.
IFRnet ducks. At least according to my tests which were sd generated animations mostly.
