NVEnc
NVEnc copied to clipboard
rigaya
[Feature Request] NVIDIA RTX Video HDR
-----Update 24.06----- The RTX Video SDK, which has just been initially released, seems to consolidate all AI-related video enhancement technologies
Super resolution Artifact reduction SDR to HDR tonemapping
The link is https://developer.nvidia.com/downloads/rtx/sdk/rtx_video_sdk_v1.0.2.zip
@rigaya Hope this helps!
--------old--------- NVIDIA has added AI-based SDR upgrade HDR in the new graphics driver. ~Could future NVEnc take advantage of related capabilities to do such conversion?~ Still seem to be waiting for NVIDIA to release a new version of the VFX SDK
Also, NVEnc requires CUDA interface for image filters (not DX12 like games), therefore I think it shall be possible whenever it is added to MAXINE-VFX-SDK. I think we'll need to wait some time for that.
Rtx Video hdr SDK 1.0 aviable,
OS: 64-bit Win10+ API support: DX11, DX12, Vulcan, (CUDA supported with DX12 interop) Supported GPUS: GeForce RTX 20 series or newer. (Turing) NVIDIA RTX 1000 or higher
I've took a loot at RTX Video hdr SDK, but it seems little difficult for me as it lacks CUDA API and requires DX11 interop.
I'll try adding this, but as I'm not bright in DX11, I think it'll take some time (few month or more) to add this to NVEnc.
NVEnc 7.56 adds RTX Video SDK features (RTX Video Super Resolution and RTX Video HDR effects) via DX11 API.
Requires Windows 550.85 driver or later and RTX 20xx GPUs.
SDR2HDR does not seem to work for me:
ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf scale=1920:1056 -color_primaries bt709 -color_trc bt709 -colorspace bt470bg -color_range tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -sws_flags spline -f yuv4mpegpipe - | NVEncC --y4m -i - --fps 25.000 --codec h265 --profile main --level auto --tier high --sar 1:1 --lookahead 32 --output-depth 10 --vbr 0 --vbr-quality 23.00 --aq --aq-strength 5 --aq-temporal --gop-len 0 --ref 6 --multiref-l0 3 --multiref-l1 3 --bframes 3 --bref-mode auto --no-b-adapt --lookahead 32 --mv-precision Q-pel --lookahead-level 0 --preset quality --colorrange limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --vpp-ngx-truehdr contrast=100,saturation=100,middlegray=50,maxluminance=1000 --cuda-schedule sync --output "g:\Output\sd2hdr_test.265"
--------------------------------------------------------------------------------
g:\Output\sd2hdr_test.265
--------------------------------------------------------------------------------
NVEncC (x64) 7.57 (r2924) by rigaya, Jun 29 2024 14:09:42 (VC 1929/Win)
OS Version Windows 11 x64 (22631) [UTF-8]
CPU AMD Ryzen 9 7950X 16-Core Processor [5.52GHz] (16C/32T)
GPU #0: NVIDIA GeForce RTX 4080 (9728 cores, 2505 MHz)[PCIe4x16][555.99]
NVENC / CUDA NVENC API 12.2, CUDA 12.5, schedule mode: sync
Input Buffers CUDA, 44 frames
Input Info y4m(yv12(10bit))->p010 [AVX2], 1920x1056, 25/1 fps
Vpp Filters copyHtoD
cspconv(p010 -> yv12(16bit))
ngx-truehdr: colorspace: cspconv(yv12(16bit) -> yuv444(16bit))
matrix:bt709->GBR
cspconv(rgb(fp32) -> rgb32)
ngx-truehdr: contrast: 100, saturation: 100, middleGray: 50, maxLuminance: 1000
cspconv(rgba(fp16) -> rgb(fp32))
colorspace: prim:bt709->bt2020
transfer:linear->smpte2084
matrix:GBR->bt2020nc
cspconv(yuv444(16bit) -> p010)
Output Info H.265/HEVC main10 @ Level auto
1920x1056p 1:1 25.000fps (25/1fps)
Encoder Preset quality
Rate Control VBR
Multipass none
Bitrate 0 kbps (Max: 28800 kbps)
Target Quality 23.00
QP range I:0-63 P:0-63 B:0-63
QP Offset cb:0 cr:0
VBV buf size auto
Split Enc Mode auto
Lookahead on, 32 frames, Level 0, Adaptive I Insert
GOP length 250 frames
B frames 3 frames [ref mode: middle]
Ref frames 6 frames, MultiRef L0:3 L1:3
AQ on (spatial, temporal, strength 5)
CU max / min auto / auto
VUI matrix:bt2020nc,colorprim:bt2020,transfer:smpte2084,range:limited
Others mv:Q-pel
encoded 429 frames, 186.20 fps, 4461.99 kbps, 9.13 MB
encode time 0:00:02, CPU: 1.5%, GPU: 63.0%, VE: 35.7%, GPUClock: 2605MHz, VEClock: 2080MHz
frame type IDR 2
frame type I 2, total size 0.10 MB
frame type P 108, total size 4.03 MB
Output has no HDR data:
MediaInfo.exe --full g:\Output\sd2hdr_test.265
General
Count : 349
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Count of video streams : 1
Video_Format_List : HEVC
Video_Format_WithHint_List : HEVC
Codecs Video : HEVC
Complete name : g:\Output\sd2hdr_test.265
Folder name : g:\Output
File name extension : sd2hdr_test.265
File name : sd2hdr_test
File extension : 265
Format : HEVC
Format : HEVC
Format/Info : High Efficiency Video Coding
Format/Url : http://www.itu.int
Format/Extensions usually used : hevc h265 265
Commercial name : HEVC
Internet media type : video/H265
File size : 9570968
File size : 9.13 MiB
File size : 9 MiB
File size : 9.1 MiB
File size : 9.13 MiB
File size : 9.128 MiB
Frame rate : 25.000
Frame rate : 25.000 FPS
File creation date : 2024-06-30 08:18:30.218 UTC
File creation date (local) : 2024-06-30 10:18:30.218
File last modification date : 2024-06-30 08:18:32.555 UTC
File last modification date (local) : 2024-06-30 10:18:32.555
Video
Count : 390
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
Format : HEVC
Format : HEVC
Format/Info : High Efficiency Video Coding
Format/Url : http://www.itu.int
Commercial name : HEVC
Format profile : Main 10@L5@High
Internet media type : video/H265
Width : 1920
Width : 1 920 pixels
Height : 1056
Height : 1 056 pixels
Sampled_Width : 1920
Sampled_Height : 1056
Pixel aspect ratio : 1.000
Display aspect ratio : 1.818
Display aspect ratio : 16:9
Frame rate : 25.000
Frame rate : 25.000 FPS
FrameRate_Num : 25
FrameRate_Den : 1
Color space : YUV
Chroma subsampling : 4:2:0
Chroma subsampling : 4:2:0
Bit depth : 10
Bit depth : 10 bits
colour_description_present : Yes
colour_description_present_Source : Stream
Color range : Limited
colour_range_Source : Stream
Color primaries : BT.2020
colour_primaries_Source : Stream
Transfer characteristics : PQ
transfer_characteristics_Source : Stream
Matrix coefficients : BT.2020 non-constant
matrix_coefficients_Source : Stream
NVIDIAs SDR2HDR option does work if I enable it in the NVIDIA Control Center and use it with a MediaPlayer. (enabling the option in the nvidia control center and then doing the encoding doesn't change a thing, output has no HDR data)
SDR2HDR does not seem to work for me:
ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf scale=1920:1056 -color_primaries bt709 -color_trc bt709 -colorspace bt470bg -color_range tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -sws_flags spline -f yuv4mpegpipe - | NVEncC --y4m -i - --fps 25.000 --codec h265 --profile main --level auto --tier high --sar 1:1 --lookahead 32 --output-depth 10 --vbr 0 --vbr-quality 23.00 --aq --aq-strength 5 --aq-temporal --gop-len 0 --ref 6 --multiref-l0 3 --multiref-l1 3 --bframes 3 --bref-mode auto --no-b-adapt --lookahead 32 --mv-precision Q-pel --lookahead-level 0 --preset quality --colorrange limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --vpp-ngx-truehdr contrast=100,saturation=100,middlegray=50,maxluminance=1000 --cuda-schedule sync --output "g:\Output\sd2hdr_test.265" -------------------------------------------------------------------------------- g:\Output\sd2hdr_test.265 -------------------------------------------------------------------------------- NVEncC (x64) 7.57 (r2924) by rigaya, Jun 29 2024 14:09:42 (VC 1929/Win) OS Version Windows 11 x64 (22631) [UTF-8] CPU AMD Ryzen 9 7950X 16-Core Processor [5.52GHz] (16C/32T) GPU #0: NVIDIA GeForce RTX 4080 (9728 cores, 2505 MHz)[PCIe4x16][555.99] NVENC / CUDA NVENC API 12.2, CUDA 12.5, schedule mode: sync Input Buffers CUDA, 44 frames Input Info y4m(yv12(10bit))->p010 [AVX2], 1920x1056, 25/1 fps Vpp Filters copyHtoD cspconv(p010 -> yv12(16bit)) ngx-truehdr: colorspace: cspconv(yv12(16bit) -> yuv444(16bit)) matrix:bt709->GBR cspconv(rgb(fp32) -> rgb32) ngx-truehdr: contrast: 100, saturation: 100, middleGray: 50, maxLuminance: 1000 cspconv(rgba(fp16) -> rgb(fp32)) colorspace: prim:bt709->bt2020 transfer:linear->smpte2084 matrix:GBR->bt2020nc cspconv(yuv444(16bit) -> p010) Output Info H.265/HEVC main10 @ Level auto 1920x1056p 1:1 25.000fps (25/1fps) Encoder Preset quality Rate Control VBR Multipass none Bitrate 0 kbps (Max: 28800 kbps) Target Quality 23.00 QP range I:0-63 P:0-63 B:0-63 QP Offset cb:0 cr:0 VBV buf size auto Split Enc Mode auto Lookahead on, 32 frames, Level 0, Adaptive I Insert GOP length 250 frames B frames 3 frames [ref mode: middle] Ref frames 6 frames, MultiRef L0:3 L1:3 AQ on (spatial, temporal, strength 5) CU max / min auto / auto VUI matrix:bt2020nc,colorprim:bt2020,transfer:smpte2084,range:limited Others mv:Q-pel encoded 429 frames, 186.20 fps, 4461.99 kbps, 9.13 MB encode time 0:00:02, CPU: 1.5%, GPU: 63.0%, VE: 35.7%, GPUClock: 2605MHz, VEClock: 2080MHz frame type IDR 2 frame type I 2, total size 0.10 MB frame type P 108, total size 4.03 MBOutput has no HDR data:
MediaInfo.exe --full g:\Output\sd2hdr_test.265 General Count : 349 Count of stream of this kind : 1 Kind of stream : General Kind of stream : General Stream identifier : 0 Count of video streams : 1 Video_Format_List : HEVC Video_Format_WithHint_List : HEVC Codecs Video : HEVC Complete name : g:\Output\sd2hdr_test.265 Folder name : g:\Output File name extension : sd2hdr_test.265 File name : sd2hdr_test File extension : 265 Format : HEVC Format : HEVC Format/Info : High Efficiency Video Coding Format/Url : http://www.itu.int Format/Extensions usually used : hevc h265 265 Commercial name : HEVC Internet media type : video/H265 File size : 9570968 File size : 9.13 MiB File size : 9 MiB File size : 9.1 MiB File size : 9.13 MiB File size : 9.128 MiB Frame rate : 25.000 Frame rate : 25.000 FPS File creation date : 2024-06-30 08:18:30.218 UTC File creation date (local) : 2024-06-30 10:18:30.218 File last modification date : 2024-06-30 08:18:32.555 UTC File last modification date (local) : 2024-06-30 10:18:32.555 Video Count : 390 Count of stream of this kind : 1 Kind of stream : Video Kind of stream : Video Stream identifier : 0 Format : HEVC Format : HEVC Format/Info : High Efficiency Video Coding Format/Url : http://www.itu.int Commercial name : HEVC Format profile : Main 10@L5@High Internet media type : video/H265 Width : 1920 Width : 1 920 pixels Height : 1056 Height : 1 056 pixels Sampled_Width : 1920 Sampled_Height : 1056 Pixel aspect ratio : 1.000 Display aspect ratio : 1.818 Display aspect ratio : 16:9 Frame rate : 25.000 Frame rate : 25.000 FPS FrameRate_Num : 25 FrameRate_Den : 1 Color space : YUV Chroma subsampling : 4:2:0 Chroma subsampling : 4:2:0 Bit depth : 10 Bit depth : 10 bits colour_description_present : Yes colour_description_present_Source : Stream Color range : Limited colour_range_Source : Stream Color primaries : BT.2020 colour_primaries_Source : Stream Transfer characteristics : PQ transfer_characteristics_Source : Stream Matrix coefficients : BT.2020 non-constant matrix_coefficients_Source : StreamNVIDIAs SDR2HDR option does work if I enable it in the NVIDIA Control Center and use it with a MediaPlayer. (enabling the option in the nvidia control center and then doing the encoding doesn't change a thing, output has no HDR data)
Could you clarify exactly what form of HDR data you are expecting.
Or what you expect HDR data to look like in Mediainfo.
It seems to me that the basic HDR eotf information(Transfer characteristics : PQ) is already contained in mediainfo at the moment.
For Mastering Display Metadata which is defined by smpte2086, this part will help.
--vpp-ngx-truehdr does not add any kind of HDR metadata, it is SDR to HDR tone mapping filter.
However it should be already tone mapped to color values sufficient for bt2020 colormatrix, and PQ transfer characteristics.
It seems to me that the basic HDR eotf information(Transfer characteristics : PQ) is already contained in mediainfo at the moment.
Naturally, since I configured them.
--vpp-ngx-truehdr does not add any kind of HDR metadata, it is SDR to HDR tone mapping filter.
Ah, okay I misunderstood what it does, I assumed that the HDR metadata would be set automatically.
Thank you for adding this feature. SDR2HDR works fine with my hardware.
Not quite real "true" HDR though, it's just a filter that copies the same SDR data into HDR. This is the same as the real-time RTX Video HDR a filter that puts same SDR into HDR by lifting the contrast, placing the color in RGB, adding saturation and pushing highlight/specular into HDR levels with same original SDR limited detail. It's just a filter like @rigaya said. Fake HDR.
