rav1e icon indicating copy to clipboard operation
rav1e copied to clipboard

Implement support for encoding Dolby Vision from RPU file

Open quietvoid opened this issue 2 years ago • 3 comments

The changes allow to encode AV1 files with Dolby Vision metadata for playback on supported devices.

As there is no official specification, the minimum I've tested is that the metadata is recognized and used correctly on playback. The metadata OBU placement is based on #3000 (the AV1 HDR10+ specification), and I've tested it to be working fine with 2 GOPs.

From CLI, the metadata is expected to be passed as a RPU binary file, which follows the same format as encoders like x265. The metadata parsing/encoding is done through the dolby_vision crate. The CLI opt can be either --dovi-rpu or --dolby-vision-rpu (used by x265)

From Rust, the metadata must be encoded into the final T.35 and provided for the frames that require it.

For muxing, it's possible to use the GPAC utilities but it currently requires patching for AV1: https://github.com/gpac/gpac/issues/2549 mkvmerge will also have support for raw OBU. IVF has to be added.

quietvoid avatar Aug 07 '23 15:08 quietvoid

There are some issues with the current T35 code..

  • With --keyint 1, some frames are missing metadata because the input frame number is incorrect when fetching the metadata from t35_q.
  • With inter frames, the lookahead causes some frame invariants to be created twice for the same frame number, with the second time having t35_metadata overwritten since the metadata has been removed from t35_q on the first time.

I don't know if the changes I made are any correct but they seem to fix the issues for me. But the metadata is still incorrectly ordered.

quietvoid avatar Aug 08 '23 00:08 quietvoid

Codecov Report

Patch coverage is 54.54% of modified lines.

Files Changed Coverage
src/bin/common.rs 12.00%
src/bin/rav1e.rs 59.09%
src/encoder.rs 76.92%
src/api/util.rs 90.00%
src/api/internal.rs 100.00%

:loudspeaker: Thoughts on this report? Let us know!.

codecov[bot] avatar Aug 08 '23 09:08 codecov[bot]

There doesn't seem to be an easy way of fixing the T35 metadata without cloning from the queue, and removing later. The alternative would be breaking the API and fixing the FrameInvariants by setting t35_metadata prior to actually encoding the packet instead of in build_frame_properties.

quietvoid avatar Aug 08 '23 14:08 quietvoid