Add Nvidia Video Effects SDK

[HunterAP23]

Description

Nvidia is releasing their Video Effects SDK. The SDK includes precompiled Windows binaries of the sample applications, but they are fulyl featured. The SDK includes the following features:

Shared Features

• All the apps support a "progress" printout for each given input.

Denoising

• Supposed to be used for webcams, but can be applied to any image or video input
• Has possible values:
  • "Weak" denoising using a value of 0
  • "Strong" denoising using a value of 1

Artifact Reduction

• Attempts to remove artifacts from low bitrate input
• Includes removing things such as blocking, ringing, mosquito noise, etc.
• Uses a float value from 0 to 1 in increments of 0.1
• Default value is 0.4

Upscaler & SuperRes

• Can upscale the input image/video by multiples of 4/3 (1.3333 repeating), 1.5, 2, 3, and 4

Here are the specific apps and how they operate:

• DenoiseEffectApp: sole purpose is to denoise given inputs
• VideoEffectsApp:
  • Can use one possible effect at a time on a given input
    • Upscaler only upscales the image - there is no denoising/artifact reduction applied
    • SuperRes upscales and allows you to apply artifact removal
    • ArtifactReduction: self explanatory
    • Denoising: self explanatory
• BatchEffectApp/BatchDenoiseEffectApp: applies denoising to multiple images in a batch method
• BatchEffectApp/BatchEffectApp: applies any of the filters available from VideoEffectsApp in a batch method.

Sources

Repository can be found here. Documentation is in that repository, but a direct can be found here.

Notes

The Nvidia Maxine repository uses the MIT license, which may conflict with Video2x's GPL v3 license.

[k4yt3x]

That's interesting to know. Thanks for sharing this. We are looking into rebuilding this project with a frame server though, so I'm not sure if it is still possible to integrate this.

[k4yt3x]

With that said, I'll keep this in mind as well.

[HunterAP23]

Yeah it might be a moot point with the change to using a frame server. The one thing about these apps is that you can run a video through them rather than individual frame images, so maybe they could be used in the future as post-processing after upscaling the input videos through the planned frame server (pretty much just denoising & artifact reduction).

Similar to this, Nvidia also has their NGX toolkit (seems to be what the filters in their Nvidia Broadcast app is based on) that also can do upscaling (multiples of 2, 4, and 8) but the current version is pretty broken where the samples don't even seem to work. This has one app for image upscaling (isr.exe which stands for Image Super resolution) and another for video upscaling (vsr.exe which stands for Video Super Resolution)

The more interesting one from the NGX toolkit is the slomo app that's supposed to interpolate given footage. Right now it's just as broken as the rest of the NGX apps, and it also only allows users to specify how many frames to insert between existing frames (IE: insert 1 frame means a 30 fps video will become 60 fps).

Figured I'd provide some info on all of it and let you look into it whenever you have free time.

[mirh]

MIT has no problems getting merged into GPL stuff.