cain-ncnn-vulkan
cain-ncnn-vulkan copied to clipboard
nihui
Having Frames with Artifacts (rgb-like)
Hi!
For some odd reason, no matter the settings (as far as i tested) i get various frames (where a blend from scene to scene would appear) where i get artifacts like these:
The more i increase the tile size, the less it gets. Issue here is that tile size 2048 seems to be the limit and nothing changes after that anymore.
Does anyone have any idea how to fix that up?
Yes CAIN uses CUDA which is Nvidia only.
It's normal for any interpolation software to output weird results for scene changes since it's trying to interpret motion where there is none. You should be using the highest tilesize you can that doesn't give vk errors so that you get the best speed/quality, -t 2048 is already bigger then 1920x1080 so higher numbers wont change anything
That I can get behind, but shouldn't it affect the entire frame then? I think I'll let that clip run through dain to see if there are similar results, because i think there's something off. That, or I'll try to find another solution which still interpolates well, but might be more artifacting. Dunno which ones to pick from though.
The frames you posted have blocks of 512x512 so either you used -t 512 or something is broken
thats what i was guessing. i did use -t 2048 though. should i set it to 1920 and test it again?
I wouldn't expect it to make a difference but you could try. 512 is the default tilesize so it seems like the -t option is getting ignored for some reason
I trained a new CAIN model, but I don’t know how to convert the pytorch model to the ncnn model
I used CAIN to make up a video, here is the link: https://drive.google.com/file/d/1w-4oU9e-LSU2JSCSbulX0Dueqdiv2Czc/view?usp=sharing
I trained a new CAIN model, but I don’t know how to convert the pytorch model to the ncnn model
https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx
https://drive.google.com/file/d/1gazwEvXVVEzAwSORoMzPhihj1ylfQzVN/view?usp=sharing
Just found out, on the same hardware and same video, i didnt experience it in linux
vimeo90k 300epoch https://drive.google.com/file/d/1yXISQLCr15LU0pDzsZMrZQzSLHJgyz1K/view?usp=sharing
@YiWeiHuang-stack how does one use that trained model on cain-ncnn? i dont seem to understand how it would be implemented.
Bumping up this issue - the problem still exists, sadly. Could you check if new NCNN versions or anything, really, improves on this issue?
It seems to be the resolution - since I have choose a resolution divide able by 128 it worked for me.
And having a monitor connected to each GPU you want to use also solved it for me.
Im pretty sure that 1920x1080 and 1280x720 should be supported. Happened across each and every file I have tested, across several OS. Glad it work for you, though!