Support additional sample formats (i.e., codecs) in RTSP

[claincly]

Use case description

Support extra sample formats with RTSP. It currently only supports

• AVC
• AAC
• AC3

Proposed solution

Add RtpPlayloadReader instances to handle more formats.

Requested format

(empty for now)

[srayhunter]

Requested format

Audio: g711 and opus.

[EugeneDubovikov]

Video: H.265 HEVC

[craftcreative]

Audio: opus

[SreeniMannem]

Will ExoPlayer supports below formats? Video: H.265 Audio: PCMA

if not, how to stream with these formats?

[goffioul]

MP2T payload type. See https://github.com/google/ExoPlayer/issues/9400.

[kurthuwig]

D-Link 2330L Camera

H264 + PCMU

v=0
o=- 1652166694306755 1 IN IP4 0.0.0.0
s=RTSP/RTP stream 1 from DCS-2330L
i=live1.sdp with v2.0
t=0 0
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:RTSP/RTP stream 1 from DCS-2330L
a=x-qt-text-inf:live1.sdp
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:1500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42001F;sprop-parameter-sets=Z0IAH5WoFAFuQA==,aM48gA==
a=control:trackID=1
m=audio 0 RTP/AVP 0
c=IN IP4 0.0.0.0
b=AS:64
a=control:trackID=2
RTSP/1.0 200 OK
CSeq: 5
Date: Tue, May 10 2022 10:55:53 GMT
Transport: RTP/AVP/TCP;unicast;interleaved=0-1;ssrc=6619AFAE
Session: 8;timeout=80

[nightcap79]

RTSP streamed via DVR Video: hevc (Main), yuvj420p(pc, bt709), 2528x1920, 20 tbr, 90k tbn Codec: MPEG-H Part2/HEVC (H.265) (hevc) Audio: pcm_mulaw, 8000 Hz, 1 channels, s16, 64 kb/s Codec: PCM MU-LAW (mlaw)

[claincly]

@nightcap79 ExoPlayer should already support HEVC and uLaw now, can you give it a try?

[nightcap79]

@claincly I'm using ExoPlayer inside Flutter via Video player plugin, but I receive the following error: I/ExoPlayerImpl(29105): Init 7391ec4 [ExoPlayerLib/2.18.1] [ginkgo, Redmi Note 8, Xiaomi, 30] E/ExoPlayerImplInternal(29105): Playback error E/ExoPlayerImplInternal(29105): com.google.android.exoplayer2.ExoPlaybackException: Source error E/ExoPlayerImplInternal(29105): at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:632) E/ExoPlayerImplInternal(29105): at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:604) E/ExoPlayerImplInternal(29105): at android.os.Handler.dispatchMessage(Handler.java:102) E/ExoPlayerImplInternal(29105): at android.os.Looper.loop(Looper.java:236) E/ExoPlayerImplInternal(29105): at android.os.HandlerThread.run(HandlerThread.java:67) E/ExoPlayerImplInternal(29105): Caused by: com.google.android.exoplayer2.upstream.HttpDataSource$HttpDataSourceException: java.net.MalformedURLException: unknown protocol: rtsp I read that since ExoPlayer 2.18.0 hevc in rtsp is working, but it seems not.

[claincly]

@nightcap79 can you file a separate issue on this? I'm not sure if ExoPlayer is setup correctly in your case, since we don't use HttpDataSource when playing back RTSP. In the issue please include your setup code for ExoPlayer.

[NicolasGodfather]

what about "ulaw" ?

[claincly]

what about "ulaw" ?

I would guess it's supported as g711-mlaw here?

https://github.com/google/ExoPlayer/blob/cf49175f790dc4bbee85bb195c021ea40765d84b/library/rtsp/src/main/java/com/google/android/exoplayer2/source/rtsp/reader/DefaultRtpPayloadReaderFactory.java#L48

[tobias-indico]

AAC LATM (I have to use VLC instead)

[claincly]

@tobias-indico AAC LATM should already be supported

https://github.com/google/ExoPlayer/blob/5d4b70666f26273fbd4c3f6dccb18ccd91093d83/library/rtsp/src/main/java/com/google/android/exoplayer2/source/rtsp/reader/DefaultRtpPayloadReaderFactory.java#L43-L44

[tobias-indico]

I still have some problems with it, but I see now that it should be supported.

Could it be that support for all types of AAC LATM is not fully implemented, and that is why the docs specifically only mentions ADTS support?

Summary of my issue with LATM audio:

• I see that ExoPlayer registers the audio codec as "mp4a.40.30" which is not a common type, however if the whole video in my RTSP stream is buffered it will play WITHOUT audio (but not until the whole video is buffered).

• If the audio codec is the simple and common "mp4a.40.2 (AAC LC)" it plays immediately as expected (with audio).

Edit/update: I have now also tested with mp4a.40.1 which has the same issue as mp4a.40.30. I see that "payloadFormat.mediaEncoding" (from the code-snippet provided above) is MP4A-LATM the times it don't work (RtpMp4aReader), and when the codec is mp4a.40.2 it is the "else" that triggers (RtpAacReader)

[itsthe1-apps]

I'm unable to play h264 mpeg-4 avc (part 10) using my ExoPlayer. But it works on VLC

[Wolfleader101]

I'm unable to play h264 mpeg-4 avc (part 10) using my ExoPlayer. But it works on VLC

I have the same issue - doesn't seem to support it.

[claincly]

AVC should be supported - could you provide the stream so that we could take a look into it?

if you can't share test content publicly, please send them to [email protected] using a subject in the format "Issue #1234" ("#1234" is replaced with your issue number).

[Wolfleader101]

AVC should be supported - could you provide the stream so that we could take a look into it?

if you can't share test content publicly, please send them to [email protected] using a subject in the format

"Issue #1234" ("#1234" is replaced with your issue number).

Hi, I can't share the stream as it's on a closed network, as it's for an IP camera. Both hikvision and dahua cameras. Happy to provide serial numbers for them if you need. But in general no hikvision or dahua camera has worked for me.

[claincly]

For HikVision maybe this https://github.com/androidx/media/issues/357 is related?

[Wolfleader101]

For HikVision maybe this androidx/media#357 is related?

That might be related (quite possibility is) - will have to do some more digging. I am using the .NET MAUI MediaElement controls so will have to implement it on the C# side and get back to you.

[wman1980]

RTSP without SPS/PPS data in 'fmtp' attribute. See also here