Magpie
Magpie copied to clipboard
Blinue
Add Nvidia Maxine AI Video Upscaler (RTX Video Super Resolution)
Expected behavior 预期的功能
more information and tests https://github.com/AaronFeng753/Waifu2x-Extension-GUI/issues/449
General info: https://developer.nvidia.com/maxine SDK package download: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/maxine/resources/maxine_windows_video_effects_sdk_ga (Nvidia account required to download) Usage guide: https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-system-guide/index.html Github but use the SDK download instead because that comes with everything: https://github.com/NVIDIA/MAXINE-VFX-SDK SDK guide but probably not relevant to us: https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-programming-guide/index.html
Alternative behavior (optional) 近似的功能（可选）
No response
Waifu2x-Extension-GUI is a fake semi-open-source project. Nothing valuable we could get. Currently there is no shader example. BTW, is RTX Video Super Resolution open?
the tests are taken from https://github.com/AaronFeng753/Waifu2x-Extension-GUI/issues/449 https://imgsli.com/MTQ1MTAx/0/1 https://imgsli.com/MTQ1MTA2/0/1
sdk only ☹️ In Waifu2x-Extension-GUI, for the scaling algorithm to work, you need to install the sdk from https://www.nvidia.com/en-us/geforce/broadcasting/broadcast-sdk/resources/
As I said,
Waifu2x-Extension-GUI is a fake semi-open-source project. Nothing valuable we could get.
MagpieFX is compute shader which is based on DirectX 11, that's why we should know if it could be converted into a shader at first.
After reviewing the documentation, I believe that integrating RTX Video Super Resolution is not impossible. Although it is not open source, it provides APIs to cooperate with D3D. I will give it a try.
Interesting note from someone who uses Waifu2x-Extension-GUI for video upscaling, the RTX Video Super Resolution that it uses NVIDIA Maxine for seems to use CUDA cores for processing instead on Tensor cores. I don't know if that means it will work on GPU's that don't have Tensor cores (GTX GPU's for instance), although the experience may not be good on older GPU's with fewer/slower CUDA cores (I have an RTX 3070 Ti).
After reviewing the documentation, I believe that integrating RTX Video Super Resolution is not impossible. Although it is not open source, it provides APIs to cooperate with D3D. I will give it a try.
Would be amazing to see Nvidia's RTX Video Super Resolution (aka Nvidia Maxine) added to Magpie. It does wonders even on YouTube videos playing via. Google Chrome.
@Blinue RTX超分 是否可以进一步推进？NVIDIA于上月已经发布了完整的NVIDIA RTX Video SDK：rtx_video_sdk_v1.0.2.zip，提供了完整的文档以及dx11/dx12接口，实现起来应该不会很难 顺带一提，NVIDIA RTX Video SDK中还附带了TrueHDR这一接口，可以将SDR内容提升为HDR内容，在支持HDR的显示器上能大幅提高观感，能否一并实现？
