webrtc icon indicating copy to clipboard operation
webrtc copied to clipboard

Allow multiple negotiated A/V codecs respectively

Open pougetat opened this issue 2 years ago • 5 comments

Description

Pion currently restricts the negotiated audio codec and negotiated video codec to the first exact / partial matches it finds in a remote description.

This causes issues in the following scenario:

  • Call RegisterCodec().
  • Call AddTransceiver() for multiple video codecs / multiple audio codecs.
  • Pion instance receives a remote description, call SetRemoteDescription() => This updates the list of negotiated codecs.
  • Pion instance creates an answer, call CreateAnswer().
  • Pion instance sets the local description, call SetLocalDescription => There is a conflict here between what transceivers have been configured and the negotiated codecs filtered down from the remote description.

Reference issue

Fixes #...

pougetat avatar Dec 11 '23 16:12 pougetat

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (eed2bb2) 76.47% compared to head (50e3727) 76.44%. Report is 5 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2632      +/-   ##
==========================================
- Coverage   76.47%   76.44%   -0.04%     
==========================================
  Files          87       87              
  Lines        9867     9895      +28     
==========================================
+ Hits         7546     7564      +18     
- Misses       1854     1861       +7     
- Partials      467      470       +3     
Flag Coverage Δ
go 77.98% <100.00%> (-0.05%) :arrow_down:
wasm 64.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Dec 12 '23 16:12 codecov[bot]

FWIW I am hitting this same issue when trying to receive multiple differently encoded tracks.

streamer45 avatar Dec 29 '23 18:12 streamer45

@Sean-Der Any thoughts on this? We may actually need to make similar changes on our end so it would be great to get some feedback on what could be the best way forward to send multi codec tracks.

streamer45 avatar Mar 05 '24 19:03 streamer45

Hey @streamer45

Is this what you are trying to implement? You want to have one TrackLocal that can be used to send many different codecs? Since you are forwarding media created by someone else, you don't know the codec at PeerConnection creation time?

Sean-Der avatar Mar 18 '24 13:03 Sean-Der

Hey @streamer45

Is this what you are trying to implement? You want to have one TrackLocal that can be used to send many different codecs? Since you are forwarding media created by someone else, you don't know the codec at PeerConnection creation time?

Hey @Sean-Der, to be honest I may have lost a bit of context on this one as I can't seem to reproduce the original issue (failing to negotiate the track) so maybe it was a simple misimplementation on our part.

Our use case is rather simple: we have one peer connection (sender) pushing two tracks with different encodings and then want to selectively forward one of the two to our receivers, depending on what they support.

streamer45 avatar Mar 18 '24 16:03 streamer45