Proper handling of specific DTS-HD profiles

[nift4]

[REQUIRED] Use case description

I am trying to determine in my app whether the playback format is lossless or not, and display format to the user so that they can verify if setup works correctly. However, even though https://www.iana.org/assignments/media-types/audio/vnd.dts.hd states:

profile is a required parameter.

ExoPlayer cannot detect whether a file is DTS-HD, DTS-HD High Resolution Audio or DTS-HD Master Audio profile, and uses a mime type without profile parameter for everything except LBR.

https://github.com/androidx/media/blob/ae9a2339679826f870690092509a135dffc0d7e4/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java#L90

For example, in MP4 extractor: https://github.com/androidx/media/blob/ae9a2339679826f870690092509a135dffc0d7e4/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java#L476-L477 even "dtsl" type which is "Extension Substream containing only XLL" -> lossless -> mime should be audio/vnd.dts.hd;profile=dtsma

Another example, ts extractor using DtsUtil always sets DTS Express mime type: https://github.com/androidx/media/blob/ae9a2339679826f870690092509a135dffc0d7e4/libraries/extractor/src/main/java/androidx/media3/extractor/DtsUtil.java#L516 but in ETSI TS 102 114 V1.6.1 Annex F a non-LBR format is explicitly mentioned as supported: "For DTS-HD Lossless formats, the value of [...]" which means DTS-HD MA profile is supported for sure, and I see no indication that other DTS-HD supports are not supported at all, hence this hardcoding to LBR is wrong.

Proposed solution

Edit ExoPlayer and extractors to extract and pass around accurate DTS-HD profile information in the mime type, including:

• MP4 extractor
• TS extractor
• MKV extractor

and do not omit the required profile parameter in the mime type.

Alternatives considered

N/A

[nift4]

Some amendment,

https://github.com/androidx/media/blob/ae9a2339679826f870690092509a135dffc0d7e4/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java#L92

AUDIO_DTS_X also is a somewhat misleading name because there is a variant of DTS:X that builds on DTS-HD MA (and that is significantly more widespread than DTS-UHD content). This can be observed on the kodi sample site: https://kodi.wiki/view/Samples#HD/object-based_Audio_Test_Clips Sample 18-20 are DTS-HD MA + DTS:X, while Sample 21 is DTS-UHD ffprobe also confirms this (18-20 are labeled DTS-HD MA + DTS:X, while 21 is unsupported by ffprobe - but ExoPlayer correctly determines 21 as DTS-UHD)

[tonihei]

Thanks for pointing this out. There was also some previous confusion around the DTS:X mime type in https://github.com/google/ExoPlayer/issues/9429.

Not sure if related, but there is also a pull request trying to parse DTS-HD from Matroska in https://github.com/androidx/media/pull/2485 (filed on the same day as your issue).

There is also further disconnect to what the Android platform defines internally here.

I agree it makes sense to improve the ExoPlayer extractor parsing to detect the right types (and also add the additional MIME type constants as needed). However, one thing I can't quite tell is what effect this has on any downstream handling in the Android platform, either in MediaCodec or when playing these formats via passthrough directly in AudioTrack. We generally have to choose the same MIME type definitions as the one in the Android platform I linked above to ensure we can find the correct matching codec for example, even if the MIME type is the "correct" one according to the standard.

[nift4]

I just saw, platform had a similar effort in the past already, just posting for reference: https://cs.android.com/android/_/android/platform/frameworks/base/+/9c85cfc37e14abdb4709320970cdff48a308000e