xbmc
Add support for Matroska DolbyVision
This commit adds support to Dolby Vision combined layer present on mkv files.
Description
Matroska recently added a AdditionBlockMapping to support Dolby Vision or other private metadata.
~The changes present here also need ffmpeg changes that are already present in Kodi's FFmpeg fork.~
EDIT: My mistake, the changes are still pending with FFmpeg.
Motivation and Context
This commit is important to bring support to the ever growing format of Dolby Vision in mkv files.
How Has This Been Tested?
This has been tested exclusively with the Nvidia Shield TV Pro (2019). Dolby Vision pass through is only possible with Apple TV and a few android boxes like the Nvidia Shield, or the Fire Stick.
Screenshots (if appropriate):
Types of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Clean up (non-breaking change which removes non-working, unmaintained functionality)
- [ ] Improvement (non-breaking change which improves existing functionality)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that will cause existing functionality to change)
- [ ] Cosmetic change (non-breaking change that doesn't touch code)
- [ ] None of the above (please explain below)
Checklist:
- [X] My code follows the Code Guidelines of this project
- [ ] My change requires a change to the documentation, either Doxygen or wiki
- [ ] I have updated the documentation accordingly
- [X] I have read the Contributing document
- [ ] I have added tests to cover my change
- [ ] All new and existing tests passed
Matroska recently added a AdditionBlockMapping to support Dolby Vision or other private metadata. The changes present here also need ffmpeg changes that are already present in Kodi's FFmpeg fork.
No they are not merged into Kodi's FFmpeg. The PR is still waiting and won't be merged until something is sent to upstream FFmpeg.
See, https://github.com/xbmc/FFmpeg/pull/18
I'll try to create a build for my firestick based on this, and report back. Unless you have an APK close to ready, will keep an eye on the forums. Edit: Welp, Maven's build is for armv8. I can't personally make MP4s work anyways: https://github.com/xbmc/xbmc/issues/18890
@quietvoid test builds are here https://mirrors.kodi.tv/test-builds/android/arm64-v8a/
Happy to announce that it doesn't work on the Firestick, just like MP4. It looks to be the same problem as the issue I created: #18890
The decoder used is correct but the TV never switches to Dolby Vision. MP4: https://paste.kodi.tv/iyikadamag.kodi MKV: https://paste.kodi.tv/axebenuhex.kodi
Hi, I made some tests with the different Dolby Vision test files
- mkv hosted by MakeMKV: https://www.makemkv.com/download/dvtest/
- mp4 hosted by Dolby: https://developer.dolby.com/tools-media/sample-media/video-streams/dolby-vision-streams/
I wanted to find out, what already works stable and how to reproduce the different issues (black screen and wrong colors). The test files are copied to my nas and accessable via a smb share within kodi file manager.
My hardware / software:
- Nvidia Shield 2019 Pro: 8.2.1
- LG OLED65C97LA: 05.00.10
- Sonos Arc: 12.2.2
The Shield is connected to HDMI 1 of my LG and the Sonos Arc to HDMI 2 (eARC Port).
- APK from Kodi Forum User "sk83": kodi19_beta_dv-arm64-v8a-advancedsettings.apk Download: https://ufile.io/76xbe039
advancedsettings.xml looks as follows:
<advancedsettings>
<cache>
<buffermode>1</buffermode>
<memorysize>157286400</memorysize>
<readfactor>20</readfactor>
</cache>
<dv_decoder>
<dvProfile7Format>amc-dvhe</dvProfile7Format>
<dvProfile8Format>amc-dvh1</dvProfile8Format>
</dv_decoder>
<dequeue_timeout>
<inputbuffer>25000</inputbuffer>
<outputbuffer>25000</outputbuffer>
</dequeue_timeout>
<loglevel>1</loglevel>
<setting id="debug.extralogging">true</setting>
<setting id="debug.setextraloglevel">128</setting>
</advancedsettings>
I combined my own settings with the settings given by @MrMagic from the Kodi forum. Thanks for the input.
My Android settings:
Settings - Device Preferences - Display & Sound - Advanced display settings
- Match content colour space -> enabled
- Custom display mode -> 3840x2160 59.940 Hz, YUV 422 12-bit Rec. 2020, Dolby Vision and HDR10 ready
Settings - Device Preferences - Display & Sound - Advanced sound settings
- Play comfort noise on HDMI -> enabled
- Dolby audio processing -> enabled
- Select formats -> Auto
My Kodi settings:
Settings - Player – Videos
- Adjust display refresh rate -> On start / stop
- Sync playback to display -> disabled
- Render method -> Auto detect
- Enable HQ scalers for scaling above -> 100% (so the Kodi scalers are not used and I can use the Shield's AI upscaling)
- Allow HA - Mediacodec (Surface) -> enabled
- Allow HA - Mediacodec -> enabled
- Use display HDR capabilities -> enabled
Settings - System – Display
- Resolution 3840x2160p
- Refresh Rate -> 59,94
- Delay after change of refresh rate -> 3.5 seconds
- Whitelist: -> 3840x2160p 60.00Hz -> 3840x2160p 59.94Hz -> 3840x2160p 50.00Hz -> 3840x2160p 24.00Hz -> 3840x2160p 23.98Hz
- Allow 3:2 pulldown refresh rates -> disabled
- Allow double refresh rates -> enabled
Settings - System – Audio
- Number of channels -> 5.1
- Allow passthrough -> enableded
- Allow passthrough: -> AC3 -> E-AC3 -> True-HD -> My Sonos Arc doesn't support DTS or DTS-HD, but it gives me PCM/LPCM instead.
My test scenarios and results:
The goal was to find out, what I have to do, to start the test files 50 times successful for some seconds (less than 5 seconds). If I find other issues like wrong colors or black screen, I want to try to make them reproducable.
| test file (profile) | Format | Resolution@FPS | Description of the result with Kodi | Description of the result with Emby |
|---|---|---|---|---|
| P4.mkv | dvhe.04.06, BL+EL+RPU, SDR compatible | 3840x2160@23,976 | To start and play the P4 file 50 times, I had to repair 61 faulty P4 starts by playing the P81 mkv test file | The P4 file started 50 times in a row. 12 times I had a color issue, which was automatically repaired with the next file start. |
| P5.mkv | dvhe.05.09, BL+RPU | 3840x2160@60,000 | It was possible to start and play the file 50 times in a row without an issue | It was possible to start and play the file 50 times in a row without an issue |
| P7FEL.mkv | dvhe.07.06, BL+EL+RPU, Blu-ray compatible / SMPTE ST 2086, HDR10 compatible | 3840x2160@23,976 | To start and play the P7FEL file 50 times, I had to repair 39 faulty P7FEL starts by playing the P81 mkv test file | The P7FEL file started 50 times in a row. 3 times it triggered HDR at my display instead of DV. With the next file start, DV was triggered again. |
| P7MEL.mkv | dvhe.07.06, BL+EL+RPU, Blu-ray compatible / SMPTE ST 2086, HDR10 compatible | 3840x2160@23,976 | To start and play the P7MEL file 50 times, I had to repair 43 faulty P7MEL starts by playing the P81 mkv test file | The P7MEL file started 50 times in a row. 5 times it triggered HDR at my display instead of DV. With the next file start, DV was triggered again. |
| P81.mkv | dvhe.08.09, BL+RPU, Blu-ray compatible / SMPTE ST 2086, HDR10 compatible | 3840x2160@59,940 | It was possible to start and play the file 50 times in a row without an issue | The P81 file didn't trigger DV, only HDR, stopped after 20 times. |
| P5.mp4 | dvhe.05.09, BL+RPU | 3840x2160@59,940 | It was possible to start and play the file 50 times in a row without an issue | It was possible to start and play the file 50 times in a row without an issue |
| P81-UHD.mp4 | dvhe.08.09, BL+RPU, HDR10 compatible / SMPTE ST 2086, HDR10 compatible | 3840x2160@59,940 | It was not possible to trigger DV at my tv once. It always started in HDR mode. Seemingly this profile is not supported in an mp4 container. | The P81-UHD file didn't trigger DV, only HDR, stopped after 20 times. The P81-FHD.mp4 triggered DV, stopped after 20 times. |
Reproducability:
My debug logs are to big (5MB per file) for paste.kodi.tv, pastebin or ubuntu paste, So I decided to generate zip files and link them here. If this is not ok for you, please tell me where to upload the logfiles. Thanks!
Faulty Start
- Go to Kodi file manager
- Start and stop the P4, the P7FEL or the P7MEL mkv test file again until it doesn’t start anymore (Happens not with P5 or P81 mkv test files)
Logfile: Started the P7FEL mkv file some times. The last start wasn’t successful.
Black Screen
- It thought I found a way to reproduce the black screen. But it was not possible. Seems to be sporadic for the moment.
Logfile: Played every mkv test file and every file triggered DV at my TVs display, but I always saw a black screen.
Wrong Colors
- Go to Kodi file manager
- Start and stop the P5 or the P7MEL mkv test file once (check if the colors look right)
- Start and stop the P4 mkv test file until it doesn’t start anymore. -> If it doesn’t start, start and stop the P81 mkv test file and start and stop the P4 mkv test file again -> Do this until the fire of the P81 mkv file is not orange anymore (I got the fire in different colors in different reproducement tests: yellow, green, purple, blue)
- Check the other test files (P5/P7FEL/P7MEL/P81) -> Now every file gives me wrong colors (even though it triggers Dolby Vision at my LG).
Logfile: Played P7FEL and P4 mkv test files alternately. The last start of the P81 file gave me a green fire.
To repair wrong colors configure the following at android os level:
Settings - Device Preferences - Display & Sound - Advanced display settings - Custom display mode
- Set to default -> enabled
- Press “OK” when the message “HDMI mode changed” appears
- Set to default -> disabled
Configure:
- Filter by resolution -> 3840x2160
- Filter by refresh rate -> 59.940 Hz
- Filter by colour space -> YUV 422 12-bit Rec. 2020, Dolby Vision and HDR10 ready
Open “Display modes”
- Choose the configured Display mode -> “3840x2160 59.940 Hz, YUV 422 12-bit Rec. 2020, Dolby Vision and HDR10 ready”
- Press “OK” when the message “HDMI mode changed” appears
Kodi UI stalls
Description: You see the Kodi file manager structure, the spinning wheel in the center of the screen, you can hear the sound, DV is not triggered at the TV. Some seconds later sound stops and the spinning wheel shows 100%, but you can still navigate in Kodi.
- Go to Kodi file manager
- Start and stop the P7FEL mkv test file until it doesn’t start -> If the P7FEL file doesn’t start, press the “back” button and start and stop the P4 mkv test file. If it also doesn’t start, press the “back” button and start and stop the P7FEL mkv test file again. -> Do this until Kodi UI stalls. Do not use the P81 mkv test file to repair a faulty start.
Logfile: Started the P7FEL file until it didn’t start and then the P4 file
My Conclusion:
- The P4/P7FEL/P7MEL mkv test files have problems to do a complete start
- The P81 mkv test file is good to repair a not starting P4/P7FEL/P7MEL mkv test file
- The P5 (mkv/mp4) and the P81 (mkv) on it’s own and not combined with other profile files start and work as expected
- It’s possible to repair wrong colors by executing the above described procedure (Is this an HDMI reset?)
- DV is always triggered at the TVs display (except when Kodi UI stalls), even though you have the issue with the black screen or the wrong colors
- As soon as you start the P5/P7FEL/P7MEL test files at first and then the P4 test file, it’s possible reproduce wrong colors
- The P4 profile seems to be really shitty.
The final question is: What is a real use case?
We don’t have P4/P5/P81 mkv content. MakeMKV generates P7FEL an P7MEL only. In my opinion we have to get the starting problem with P7FEL and P7FEL under control. No one starts files with different DV profiles this frequently in real life.
By the way… With Emby for AndroidTV you do not have the starting problems. I also played the 5 mkv test file 50 time in a row. Every start was complete. I could not reproduce the errors I (we) have with Kodi.
I hope these issues are also reproducable procedures for you.
The debug log was configured to log the FFmpeg component to hold the logging files small. If you need something different, please let me now.
Regards Hoppel
Hi,
I uploaded my debug logs as zip files. I wasn't aware of that it's possible to upload them here directly.
My debug logs are to big (5MB per file) for paste.kodi.tv, pastebin or ubuntu paste, If this is not ok for you, please tell me where to upload the logfiles. Thanks!
Regards Hoppel
Hi,
I made some further tests with the latest @fandangos dv test build:
https://mega.nz/file/XEdm2LAZ#efDt63w4JU3jxsS2uGqR5hTgL4uzlYHeqapxFvQ2Oeg
I only tried to play the P7FEL and P7MEL mkv test files, because it's the only relevant material. We do not create mkvs with P4, P5 or P81 in real life. I also checked with real P7 FEL/MEL mkv movies. The result was the same.
I deinstalled the Kodi build and installed it again to have a completely clean installation. I mainly used the default Kodi settings.
My Android settings:
Settings - Device Preferences - Display & Sound - Advanced display settings
- Match content colour space -> enabled
- Custom display mode -> 3840x2160 59.940 Hz, YUV 422 12-bit Rec. 2020, Dolby Vision and HDR10 ready
Settings - Device Preferences - Display & Sound - Advanced sound settings
- Play comfort noise on HDMI -> enabled
- Dolby audio processing -> enabled
- Select formats -> Auto
My Kodi settings:
Settings - Player – Videos (default settings)
- Adjust display refresh rate -> Off
- Sync playback to display -> disabled
- Render method -> Auto detect
- Enable HQ scalers for scaling above -> 20%
- Allow HA - Mediacodec (Surface) -> enabled
- Allow HA - Mediacodec -> enabled
- Use display HDR capabilities -> enabled
Settings - System – Display (default settings)
- Resolution 3840x2160p
- Refresh Rate -> 59,94
- Delay after change of refresh rate ->0 seconds
- Whitelist -> not configured
- Allow 3:2 pulldown refresh rates -> disabled
- Allow double refresh rates -> disabled
Settings - System – Display (customized settings)
- Number of buffers used by graphics driver -> 2
Settings - System – Audio (customized settings)
I also tried to start the mkv test files with the default audio settings. The result was the same.
- Number of channels -> 5.1
- Allow passthrough -> enableded
- Allow passthrough: -> AC3 -> E-AC3 -> True-HD -> My Sonos Arc doesn't support DTS or DTS-HD, but it gives me PCM/LPCM instead.
My Kodi advancedsettings.xml ist also stripped down to debug FFmpeg only:
<advancedsettings>
<loglevel>1</loglevel>
<setting id="debug.extralogging">true</setting>
<setting id="debug.setextraloglevel">128</setting>
</advancedsettings>
My Logfiles:
The test case procedure
- Start/Restart Kodi
- Got to Kodi File manager
- Start the file, play the file for 10 seconds if possible, stop the file -> again, again, and so on...
P7FEL:
First try to play the file P7FEL.mkv didn't work. The result is what I call a "faulty start" in my last comment above.
The same test case procedure again. The first try worked as expected. Didn't do any further start stops. I only want to show how it has to look, if it works.
P7MEL:
First try to play the file P7MEL.mkv didn't work. The result is what I call a "faulty start" in my last comment above.
The same test case procedure again. The first try worked as expected. Didn't do any further start stops. I only want to show how it has to look, if it works.
I hope this helps. If you want to see other test cases or debug components logged, please tell me what you need exactly.
Regards Hoppel
@fandangos Made a patch for a workaround for my FireTV Stick 4K as well as improving playback for some TS/MP4 files that would not try using the DVHE decoder. You can find it here: https://gist.github.com/quietvoid/db7ac49bd8621a82f914888bca730c74
It can probably be improved but that's the best I could come up with. It might be better for the workaround to be another PR, to close #18890 but the demuxer code would improve hinting Dolby Vision streams.
Is there a way to track the changes on ffmpeg side? Like a ticket, a PR or something? I searched their ticket system but didn't find anything relevant.
Is there a way to track the changes on ffmpeg side? Like a ticket, a PR or something? I searched their ticket system but didn't find anything relevant.
This is the last patch set I sent: https://patchwork.ffmpeg.org/project/ffmpeg/cover/[email protected]/ Will be sending a revised version in the coming week.
The required patches for Dolby Vision decoder config in Matroska container should now be in the next FFmpeg release, so 5.0. So a separate FFmpeg fork would no longer be necessary once that releases and Kodi updates FFmpeg.
There's now just a need for a better way to pass the "contains Dolby Vision" information from the demuxer.
Using codec_tag might end up confusing other decoders that rely on it, but it's also how inputstream.adaptive does it.
FFmpeg 5.0 released on 17/01/22, with the promised support for Dolby Vision; are there any blockers for Kodi to update its FFmpeg version to 5.0? Once that's done, I hope work on this PR can continue with minimal changes needed for merge.
The blocker now is that updating from ffmpeg 4.4 to 5.0 is not simple as lot of API's have changed or been depreciated altogether, so updating could potententially break a lots of things. However I don't believe anyone has looked in detail at it yet to fully assess the risks.
FYI FFmpeg is now at 5.1 (released July 22, 2022). Is there any chance of this making it into v20 so DV files are supported?
There seems to be little chance as updating to ffmpeg 5 requires significant reworks to the code. No one currently working on it as far as I'm aware, since most seem to view it as too big of a job at this stage in v20 development cycle.
Ah! Thanks for the update. I guess https://github.com/xbmc/xbmc/pull/21248#issuecomment-1094367811 implied that there was much work on this in progress.
#21248 looks on a good track to use FFmpeg 5.1.2 for kodi. Does this mean work on this PR could resume next?
#21248 looks on a good track to use FFmpeg 5.1.2 for kodi. Does this mean work on this PR could resume next?
Not sure if this is even still needed, or whether ffmpeg 5 on it's own will be enough. Perhaps @quietvoid knows.
Not sure if this is even still needed, or whether ffmpeg 5 on it's own will be enough. Perhaps @quietvoid knows.
The current method is to pass the information from the demuxer using the stream's codec_tag.
This however requires hardcoding conditions to match specific tags, for Matroska it was dvvC/dvcC.
So I'm not sure if it would be better to simply use CDemuxStreamVideo::hdr_type instead.
Addons like ISA still use dvhe, dvh1 though.
There's still a need to provide more info for Matroska.
#21248 (use of FFmpeg 5.1.2) has been merged to xbmc:master. So I guess the next step is to converge on the best way to pass the 'contains Dolby Vision" information for Matroska files?
#21248 (use of FFmpeg 5.1.2) has been merged to xbmc:master. So I guess the next step is to converge on the best way to pass the 'contains Dolby Vision" information for Matroska files?
All that's needed for DV on mkv is already present in this commit. Just use the code from DVDVideoCodecAndroidMediaCodec and DVDDemuxFFmpeg (thanks @lrusak) .
@quietvoid can confirm but that's about it, the code is here.
I'm going to close this PR because I can't work on it at the moment and it became really messy on my end (main branch has the PR for full bluray support on android and mkv dv).