mpv icon indicating copy to clipboard operation
mpv copied to clipboard

RTX Video Enhancement support

Open germn opened this issue 3 years ago • 42 comments

There was an RTX Video Enhancement release recently. https://nvidia.custhelp.com/app/answers/detail/a_id/5448

Basically, it allows real-time AI video upscaling on recent RTX video cards. It would be nice to have this feature available in mpv as well.

There's already a release for MPC with support of this: https://github.com/emoose/VideoRenderer/releases/tag/rtx-1.0

germn avatar Mar 01 '23 06:03 germn

There is already vf_d3d11vpp.c so it is actually trivial to extend it to select correct post processing extension. I personally don't have hardware to work with and implementing it blindly is never fun, so let's wait for another brave soul to add this.

kasper93 avatar Mar 05 '23 20:03 kasper93

Just wanted to support this idea. No doubt that a MPV hero will take on this mission. :) The new feature is supposed to work not only on 4xxx cards but on 3xxx cards also.

Caution, there may be some issues to tackle : memory leaks on some HEVC 10bit videos, or the enhancements not working on vertical videos. See here: https://www.reddit.com/r/nvidia/comments/11e7e2h/comment/jacujw9/?utm_source=reddit&utm_medium=web2x&context=3 https://github.com/emoose/VideoRenderer/issues/1#issuecomment-1450964365

JoseHidalgo avatar Mar 06 '23 14:03 JoseHidalgo

I'm totally interessed to have this feature in mpv. I tried RTX VSR on MPC-BE and almost completely beat many upscalers I use with mpv, like FSRCNNX. It is absolutely worth it.

askasys-code avatar Mar 09 '23 09:03 askasys-code

I can take a look at this after getting access to a 40 series GPU

cloud11665 avatar Mar 24 '23 10:03 cloud11665

I can take a look at this after getting access to a 40 series GPU

If you need a tester @ me, I have a 4080

Myaanee avatar Mar 24 '23 10:03 Myaanee

I can take a look at this after getting access to a 40 series GPU

As a side-note to avoid possible misconceptions: RTX Video is also available for the RTX 30xx series.

NekuSoul avatar Mar 24 '23 11:03 NekuSoul

I currently have a 2080, and I will have access to a 4070 in ~2-3 weeks

cloud11665 avatar Mar 24 '23 11:03 cloud11665

Tested on vlc-3.0.19 and it works. Now only mpv remain, I really hope it will be supported soon. Maybe with a lua script? A toggle option is mandatory in my opinion, because both MPC-HC and VLC have VSR always ON and it's annoying.

askasys-code avatar Apr 13 '23 17:04 askasys-code

yes we are waiting for MPV my RTX 3080

360hzz avatar May 08 '23 11:05 360hzz

there is already a VSR for VapourSynth https://github.com/HolyWu/vs-basicvsrpp only if someone would have build it

This doesn't seem to have anything to do with the NVIDIA VSR based on d3d11vpp.

really sorry ooooppps

saha1914 avatar May 28 '23 16:05 saha1914

update: I now have access to a 40-series GPU, just need to set up my devenv on windows, and I can take a closer look.

cloud11665 avatar Jun 05 '23 16:06 cloud11665

update: I now have access to a 40-series GPU, just need to set up my devenv on windows, and I can take a closer look.

Any luck?

deama avatar Jul 14 '23 21:07 deama

Aaaaand?

radozd avatar Oct 19 '23 14:10 radozd

Aaaaand?

Aaaaand, I can implement that, just need a GPU. We can discuss shipping/payment details in email. You can find my email address in my profile. (my nick on gmail)

kasper93 avatar Oct 19 '23 15:10 kasper93

@kasper93 RTX VSR 1.5 now works on RTX 2000 series as well, don't need 4000 series.

Holdo avatar Oct 24 '23 17:10 Holdo

Aaaaand?

Aaaaand, I can implement that, just need a GPU. We can discuss shipping/payment details in email. You can find my email address in my profile. (my nick on gmail)

So I guess this was a joke or were you trying to scam someone?

deama avatar Nov 23 '23 21:11 deama

So I guess this was a joke or were you trying to scam someone?

Scam of what? It has been months and looks like no one capable of adding this feature to mpv cares to do so. Like I said, I don't have NVIDIA GPU, nor I intend to buy one, so I won't work on this feature, unless someone contribute the GPU. And since there is constant nagging in here to get it implemented I offered my help and stated my price. I was mostly joking, because I don't expect you guys understand how contracting works, but messages like Aaaaand? won't persuade anyone to work on this feature.

Look at my recent commit history in here if you call out scams... If anything I'm scammed of my free time working on this project.

kasper93 avatar Nov 23 '23 22:11 kasper93

So I guess this was a joke or were you trying to scam someone?

Scam of what? It has been months and looks like no one capable of adding this feature to mpv cares to do so. Like I said, I don't have NVIDIA GPU, nor I intend to buy one, so I won't work on this feature, unless someone contribute the GPU. And since there is constant nagging in here to get it implemented I offered my help and stated my price. I was mostly joking, because I don't expect you guys understand how contracting works, but messages like Aaaaand? won't persuade anyone to work on this feature.

Look at my recent commit history in here if you call out scams... If anything I'm scammed of my free time working on this project.

I think it looks pretty good.

deama avatar Nov 26 '23 10:11 deama

So I guess this was a joke or were you trying to scam someone?

Scam of what? It has been months and looks like no one capable of adding this feature to mpv cares to do so. Like I said, I don't have NVIDIA GPU, nor I intend to buy one, so I won't work on this feature, unless someone contribute the GPU. And since there is constant nagging in here to get it implemented I offered my help and stated my price. I was mostly joking, because I don't expect you guys understand how contracting works, but messages like Aaaaand? won't persuade anyone to work on this feature.

Look at my recent commit history in here if you call out scams... If anything I'm scammed of my free time working on this project.

If you consider two messages between Jun and Oct as "constant nagging" and it bothers you, and you require sponsorship to start your work then probably you shouldn't spend your time for an open source project. Complains like yours won't persuade anyone away from asking for new features, explanations like @Andarwinux's will.

kenpuluma avatar Nov 28 '23 05:11 kenpuluma

If you consider two messages between Jun and Oct as "constant nagging" and it bothers you, and you require sponsorship to start your work then probably you shouldn't spend your time for an open source project. Complains like yours won't persuade anyone away from asking for new features, explanations like @Andarwinux's will.

People like you make me hate volunteering my work for open source.

Your sense of entitlement is absurd. Nobody owes you this feature. And also nobody owes you an in-depth explanation of the technical or personal or monetary or whatever challenges keep it from being implemented as fast as you might like.

The feature was asked for. The issue was not closed, so it's welcome. If you want it so much, develop it yourself and submit a PR. Otherwise, sit yourself down and wait patiently and politely and gratefully for the project's volunteers to do the work for you, free of charge, while you type "And?".

tliron avatar Nov 28 '23 05:11 tliron

If you consider two messages between Jun and Oct as "constant nagging" and it bothers you, and you require sponsorship to start your work then probably you shouldn't spend your time for an open source project. Complains like yours won't persuade anyone away from asking for new features, explanations like @Andarwinux's will.

People like you make me hate volunteering my work for open source.

Your sense of entitlement is absurd. Nobody owes you this feature. And also nobody owes you an in-depth explanation of the technical or personal or monetary or whatever challenges keep it from being implemented as fast as you might like.

The feature was asked for. The issue was not closed, so it's welcome. If you want it so much, develop it yourself and submit a PR. Otherwise, sit yourself down and wait patiently and politely and gratefully for the project's volunteers to do the work for you, free of charge, while you type "And?".

I'm not begging you for development, I'm not even the one who typed "And". I'm just searching online and happened to found this issue, and it looks to me that the discussion was normal until somebody asks for a payment (I guess it's a joke).

If you hate volunteering your work for open source then don't do it, like you said nobody is forcing you.

kenpuluma avatar Nov 28 '23 06:11 kenpuluma

Complains like yours won't persuade anyone away from asking for new features

Patches welcome

llyyr avatar Nov 28 '23 06:11 llyyr

[...] probably you shouldn't spend your time for an open source project

This is not your place to state what others should or should not do in their own time.

the discussion was normal until somebody asks for a payment (I guess it's a joke)

Perhaps you're from some communist utopia but asking for payment for your work is nothing abnormal around the rest of the world.

And if you're so bothered by someone not wanting to spend hundreds of dollars to implement a feature for free for randoms on the internet, then go ahead and be the change you wish to see in the world. Buy yourself a compatible gpu and send the patches.

N-R-K avatar Nov 28 '23 06:11 N-R-K

I really don't want to populate the Github issue with meaningless, unrelated messages in a but if you insist ok:

This is not your place to state what others should or should not do in their own time.

Agreed. Github issue is the place for tracking work and collaboration on the project, not for making jokes and spreading hates and stress.

Perhaps you're from some communist utopia but asking for payment for your work is nothing abnormal around the rest of the world.

Don't try to change the subject. It's definitely normal for asking payments for anything you have spent time on, lots of people work on open source projects for a living. But the discussion went pretty off the track and became completely irrelevant about its original topic: "RTX Video Enhancement support" now, people started defending themselves on scam or not scam, be thankful to open source contributors or not , beginning from that payment joke.

If this is how you want this repository looks like, then it's fine, leave people there to fight against each others.

kenpuluma avatar Nov 28 '23 07:11 kenpuluma

@kenpuluma

Hey. Basically I'm not going to implement it (the NVIDIA VSR) I know... UGH!! I know... I'm sorry!!!! I'm just not gonna implement it is all HAHAHAHAHAHA

ghost avatar Nov 28 '23 07:11 ghost

VSR has no status feedback, so there is no reliable way to confirm VSR is active.

How the driver handles the scaling is indeed not visible. But adding generic d3d11vpp scaling is good either way. Even if not VSR, it would still scale, deepening on vendor with various quality.

Also note that passing frames through video processor can/will also apply other adjustments that they have in pipeline, which in most cases is not wanted.

This feature was designed primarily for Chromium, which is planning to move to D3D12, so D3D11VPP VSR will certainly be dropped eventually, or even removed from drivers at some point. Adding VSR may not be worth it at this time.

I think you infer too much. d3d11 or d3d12 is only an API, there is no need to remove the scaler from d3d11 video processor to make it work for d3d12 video processor. Unless there would be needed API extension.

Intel actually also support VSR (Arc only) but like NVIDIA there is no status feedback and Intel's VSR has no gpu control panel switches or options and the effect is much weaker, you never know if VSR is active or not.

Similar situation, we just request the scaling. It is on driver control panel to expose proper feedback/control.

kasper93 avatar Nov 28 '23 08:11 kasper93

Don't listen to paid AMD shills and false flaggers trying to subvert this issue

haasn avatar Nov 28 '23 09:11 haasn

@kasper93 RTX VSR 1.5 now works on RTX 2000 series as well, don't need 4000 series.

AMD has it's own upscaling now. You don't even need NVIDIA gpu to implement this. Just the latest AMD ones. (at least for the AMD, probably different guids and so on)

AMD Video Upscaling – Advanced video upscale algorithm to improve video playback image quality for AMD Radeon™ RX 7000 desktop series GPUs.

https://www.amd.com/en/support/kb/release-notes/rn-rad-win-24-1-1

...

kasper93 avatar Jan 24 '24 00:01 kasper93

I assume this'll be the default setting for Chrome/Edge somewhat soon?

BTW, the description blurb for this flag is:

Enable/Disable the use of MediaFoundation for non-protected content playback on supported systems. – Windows

Implying that MediaFoundation is already used for Widevine content etc.

Hrxn avatar Jan 24 '24 01:01 Hrxn

MPC-HC, Potplayer and VLC has already started using RTX VSR. Can we expect to see this in MPV soon?

wadixx avatar Apr 19 '24 18:04 wadixx