obs-studio icon indicating copy to clipboard operation
obs-studio copied to clipboard

Published 20 hours ago • verified publisher icon obsproject

obs-outputs: Add eRTMP multitrack audio support

Open palana opened this issue 3 months ago • 2 comments

Description

In addition to multitrack video, eRTMP also includes support for multitrack audio, thus making things like VOD track available in the spec, rather than relying on FLV byte arrays

Motivation and Context

VOD track currently uses FLV byte arrays to carry the additional audio track, where byte arrays are part of the spec, but using them for audio isn't. With the updated eRTMP multitrack audio support general overhead is lower and also the transport format is part of an open specification.

How Has This Been Tested?

eRTMP multitrack audio is being tested internally at Twitch

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

  • [x] My code has been run through clang-format.
  • [x] I have read the contributing document.
  • [x] My code is not on the master branch.
  • [x] The code has been tested.
  • [x] All commit messages are properly formatted and commits squashed where appropriate.
  • [x] I have included updates to all appropriate documentation.

palana avatar Apr 29 '24 11:04 palana

#10500 has been merged, so this can be rebased and undrafted.

RytoEX avatar May 03 '24 17:05 RytoEX

For what it's worth, I tested this and it worked as expected/configured.

RytoEX avatar May 08 '24 21:05 RytoEX

Holding this due to a confirmed issue regarding audio/video desync that is unique to this implementation.

RytoEX avatar May 14 '24 23:05 RytoEX

Adding the link to the public alpha Enhanced RTMP v2 spec for posterity.

koolscooby avatar May 16 '24 21:05 koolscooby

Holding this due to a confirmed issue regarding audio/video desync that is unique to this implementation.

@RytoEX: I just pushed a fix for the desync, it was an issue in librtmp's handling of type 3 chunks -> undrafting

palana avatar May 21 '24 11:05 palana

Great work tracking that multitrack bug down. That must have been a pain.

Lain-B avatar May 21 '24 21:05 Lain-B

Not sure if the librtmp commit should be first or not. I wouldn't want the eRTMP multitrack audio commit to exhibit the buggy behavior, but I wasn't sure if the librtmp commit has other effects on the existing VOD Audio Track implementation.

RytoEX avatar May 21 '24 21:05 RytoEX

Not sure if the librtmp commit should be first or not. I wouldn't want the eRTMP multitrack audio commit to exhibit the buggy behavior, but I wasn't sure if the librtmp commit has other effects on the existing VOD Audio Track implementation.

agreed, will move it to the beginning

it doesn't affect the current vod track (as far as I can tell) because the current vod track uses RTMP_PACKET_TYPE_INFO ^1 which always ends up as RTMP_PACKET_SIZE_LARGE ^2 and RTMP_PACKET_SIZE_LARGE always carries an absolute timestamp and is never being considered for header compression ^3

palana avatar May 22 '24 10:05 palana

Further tests with the added librtmp commit seem promising, so I'm going to call this safe and merge it.

RytoEX avatar May 24 '24 19:05 RytoEX