LAVFilters
LAVFilters copied to clipboard
Nevcairiel
Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics
First of all I want to thank you for this great project, I'm enjoying it so much on my Windows machine.
Intel graphics have supported decoding of HEVC 4:2:2 and 4:4:4 8/10bits since Ice Lake released in 2019. But even in 2023, Microsoft has not provided any official header file and struct for the HEVC Rext. Fortunately Intel made it possible for Chromium browser and a few video players such as VLC and MPC-BE to leverage it by using the vendor-specific DXVA entries.
It would be great if we can also have this feature in LAVFilters, given that 4:2:2 is becoming more and more common in the output format of high-end SLR cameras. It's difficult for users to play them on Windows smoothly without a powerful CPU, especially when it comes to 8k 4:2:2 clips.
I read this discussion from three years ago, can we revisit the patch series? Friendly paging @Nevcairiel https://patchwork.ffmpeg.org/project/ffmpeg/patch/[email protected]/
If the spec becomes official, I'll gladly support it. Until then, not adding code without a spec reference.
Intel media developer recently pointed out in Chromium-Review that Microsoft will soon release the DXVA Rext specification, we just have to wait.
Nice to see that. They almost forgot about it.
Seems like my tiny effort is paying off :P https://github.com/intel-media-ci/ffmpeg/pull/602#issuecomment-1363673893
Intel also supports HEVC SCC, but not sure if Microsoft will also add support for this profile, a certain Intel media developer told me that at least for now there are no plans.
I don't see an urgent need for HEVC SCC hwaccel, and it wouldn't be a problem if it came later.
If the spec becomes official, I'll gladly support it. Until then, not adding code without a spec reference.
Thanks for the response!
Microsoft has released the DXVA Rext specification. The relevant GUIDs can be found in the latest Canary SDK.
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);
Microsoft has released the DXVA Rext specification. The relevant GUIDs can be found in the latest Canary SDK.
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01); DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01); DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);
@Andarwinux Great! Thanks for the heads up. cc @Nevcairiel
If there is an actual specification, a link to that would be helpful. On a quick glance I couldn't find it, but I didn't have much time to go digging right now.
The online specification documentation is not yet available. Microsoft will update the stable Windows SDK shortly, and the documentation will be updated at that time. But for now the new headers needed for Rext support are available from the Canary Windows SDK, and Chromium developers will be writing a new implementation of Chromium D3D12 HEVC Rext next week.
Update: DirectX-Headers already contain new redistributable headers. https://github.com/microsoft/DirectX-Headers/blob/main/include/directx/d3d12video.h#L6304
Apparently they ship preview SDK on nuget https://www.nuget.org/packages/Microsoft.Windows.SDK.CPP.x64/10.0.25936-preview (includes those new defines), but I feel like there is no point in doing anything until there is proper release and documentation.
At least the latest preview SDK has all the required information - however no drivers exist yet that support it, at least for non-insider windows (and I'm not bothering with that). So it might only become available once the next major update of Windows 11 rolls out.
At least the latest preview SDK has all the required information - however no drivers exist yet that support it, at least for non-insider windows (and I'm not bothering with that). So it might only become available once the next major update of Windows 11 rolls out.
Perhaps we add support by first borrowing the GUID already implemented in Intel's existing driver, and then switch when the new driver & GUIDs becomes available in Windows 11? Driver support for some early Rext-enabled Intel graphics cards such as 10th Gen Ice Lake and Jasper Lake has been moved to maintenance-only status. They may never receive drivers containing these new GUIDs, but their Intel-specific GUIDs are always available.
https://github.com/hughbe/windows-sdk-headers/blob/89f151d343587fcd5b854a4851fcb8f7187f3ac4/Microsoft.Windows.SDK.CPP/latest/Include/um/dxva.h#L1273-L1304
Still, I haven't carefully compared the differences between the old and new DXVA_PicParams_HEVC_RangeExt structures. Correct me if I'm wrong.
Unfortunately the Intel version of that structure is not compatible with the Microsoft version, so I won't be offering support for those.
Unfortunately the Intel version of that structure is not compatible with the Microsoft version, so I won't be offering support for those.
That makes sense. They are just incompatible.
It seems that HW decoding for HEVC 444 is now working using the D3D11 decoder in Potplayer. I'm using latest public Intel driver version 5522(https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). So I assume the driver is no longer a problem. Hope you guys can get it working soon.
"MPC Video Renderer" uses Intel-specific HEVC Rext GUIDs.
Well ,I switched to D3D11 Video Render this time, and it seems to still be using HW decode.
PotPlayer, as a closed source software, also supported Intel-specific GUIDs not long ago.
Intel actually supports a good chunk of the new GUIDs already, but there is still no official spec on their limitations etc, which I expect will arrive with the next Windows 11 major update (24H2), which will be coming out sometime in the second half of this year.
-> HEVC / H.265 variable-length decoder, main
-> HEVC / H.265 variable-length decoder, main10
-> HEVC / H.265 variable-length decoder, main10 extended
-> HEVC / H.265 variable-length decoder, main10 422
-> HEVC / H.265 variable-length decoder, main 444
-> HEVC / H.265 variable-length decoder, main10 444
-> HEVC / H.265 variable-length decoder, main12
-> HEVC / H.265 variable-length decoder, main12 422
-> HEVC / H.265 variable-length decoder, main12 444
https://github.com/jellyfin/jellyfin-ffmpeg/pull/202 https://code.videolan.org/videolan/vlc/-/commit/b693265a387f74e39cbe9d8cd37d0c0190ee4a0d This has been supported since a long time (and there was a recent attempt tried by the MPC-BE guy)
Hooking up that seems more important than whatever W11 (and Xe likely?) only path.
jellyfin/jellyfin-ffmpeg#202 https://code.videolan.org/videolan/vlc/-/commit/b693265a387f74e39cbe9d8cd37d0c0190ee4a0d This has been supported since a long time (and there was a recent attempt tried by the MPC-BE guy)
Hooking up that seems more important than whatever W11 (and Xe likely?) only path.
New generic GUIDs for Rext also appeared on W10 (UHD7xx, Xe and newer), but there is no document on how to use these interfaces. FFmpeg does not want to accept Intel-specific interface definitions. So either you patch FFmpeg yourself, or wait for Microsoft to update the documentation and developers to submit new patches.
Hooking up that seems more important than whatever W11 (and Xe likely?) only path.
An Intel-only proprietary method which is also entirely undocumented and incompatible with the official support seems not more anything.
As I've said before, the Intel-specific version will not be supported. The official specification from Microsoft is almost here, and all current Intel drivers support it on all hardware thats still getting new drivers. My UHD 770 iGPU certainly exposes it. Its also not limited to Windows 11, as all it really is, is a specification of how data is exchanged, it does not require OS support.
In any case, hopefully Microsoft updates the specification document soon to give additional guidance on how to use it, but even if they don't its not entirely impossible to implement otherwise, just will take more time figuring stuff out.