espressif
No way to modify RTSP SDP payload type (AUD-6472)
Problem Description I'm using esp-idf v5.4.0 and esp-adf at commit c3609b9c with the official ESP32-P4-Function-EV-Board. I'm streaming video using the internal RTSP server provided by ESP-ADF. However, I’ve encountered a critical limitation: There is no way to configure or override the SDP payload type or codec description returned by the RTSP server in the DESCRIBE response to client. This results in the SDP always looks like this: I (22164) ESP_RTSP_SERVER: RTSP received [DESCRIBE], len 146 I (22175) ESP_RTSP_SERVER: TX:
RTSP/1.0 200 OK CSeq: 3 Content-Type: application/sdp Content-Length: 114
v=0 o=Z 0 3 IN IP4 192.168.31.48 s=Z t=0 0 c=IN IP4 192.168.31.48 m=video 0 RTP/AVP 26 a=control:trackID=1
The payload type 26 corresponds to JPEG, while in reality my stream is encoded using H.264. As a result, standard clients like VLC misinterpret the codec and fail to decode the stream correctly. So looks like it's a few problems there: 1.There is no public API or sdp configuration hook to override the SDP content. 2.The RTSP source is closed, so there's no way to patch or modify this behavior. 3.Without proper SDP headers like rtpmap and fmtp, dynamic payload types cannot be used.
Device Logs I (10969) ESP_RTSP_SERVER: esp_rtsp ver: 1.1.0 compile date: Sep 27 2024-03:11:17 I (10970) ESP_RTSP_SERVER: RTSP server task started URL: rtsp://192.168.31.48:8554 esp32p4> I (22113) ESP_RTSP_SERVER: RTSP Client ip address: 192.168.31.31 I (22113) ESP_RTSP_SERVER: RX:
OPTIONS rtsp://192.168.31.48:8554 RTSP/1.0 CSeq: 2 User-Agent: LibVLC/3.0.20 (LIVE555 Streaming Media v2016.11.28)
I (22125) ESP_RTSP_SERVER: RTSP received [OPTIONS], len 120 I (22136) ESP_RTSP_SERVER: TX:
RTSP/1.0 200 OK CSeq: 2 Public: OPTIONS, DESCRIBE, SETUP, PLAY, TEARDOWN Server: ESP32-RTSP-Server 1.1.0
I (22152) ESP_RTSP_SERVER: RX:
DESCRIBE rtsp://192.168.31.48:8554 RTSP/1.0 CSeq: 3 User-Agent: LibVLC/3.0.20 (LIVE555 Streaming Media v2016.11.28) Accept: application/sdp
I (22164) ESP_RTSP_SERVER: RTSP received [DESCRIBE], len 146 I (22175) ESP_RTSP_SERVER: TX:
RTSP/1.0 200 OK CSeq: 3 Content-Type: application/sdp Content-Length: 114
v=0 o=Z 0 3 IN IP4 192.168.31.48 s=Z t=0 0 c=IN IP4 192.168.31.48 m=video 0 RTP/AVP 26 a=control:trackID=1
I (22203) ESP_RTSP_SERVER: RX:
SETUP rtsp://192.168.31.48:8554/trackID=1 RTSP/1.0 CSeq: 4 User-Agent: LibVLC/3.0.20 (LIVE555 Streaming Media v2016.11.28) Transport: RTP/AVP;unicast;client_port=34294-34295
I (22216) ESP_RTSP_SERVER: RTSP received [SETUP], len 180 I (22227) ESP_RTSP_SERVER: RTSP Client vrport 34294 I (22227) RTSP_SERVICE: RTSP_STATE_SETUP I (22228) ESP_RTSP_SERVER: TX:
RTSP/1.0 200 OK Transport: RTP/AVP;unicast;client_port=34294-34295;server_port=24632-24633 CSeq: 4 Session: 6416403
I (22256) ESP_RTSP_SERVER: RX:
PLAY rtsp://192.168.31.48:8554 RTSP/1.0 CSeq: 5 User-Agent: LibVLC/3.0.20 (LIVE555 Streaming Media v2016.11.28) Session: 6416403 Range: npt=0.000-
I (22268) ESP_RTSP_SERVER: RTSP received [PLAY], len 154 I (22269) H264_ENC.SW: openh264 version: v1.0.0 I (22412) AUDIO_THREAD: The _video_enc task allocate stack on external memory I (22413) RTSP_SERVICE: RTSP_STATE_PLAY I (22432) ESP_RTSP_SERVER: TX:
RTSP/1.0 200 OK CSeq: 5 Session: 6416403
Feature Request Please provide a mechanism to: Override the SDP payload type and codec description dynamically.
This functionality is essential for any compliant RTSP + H.264 streaming setup. Let me know if further logs, source modifications, or test cases are needed.
So could you please describe the detail request for RTSP server? Currently RTSP server only allow local push and others to connected in the pull the stream, it is not a general server. For general server you can use RTMP, currently RTMP server offer a general server role support pc push and pull.
@TempoTian Thank you for the clarification. My request is specifically about the RTSP server behavior: I would like to be able to customize the RTP payload type (e.g., change it from 26 to 96 or any other value appropriate for H.264). Currently, the RTSP server assigns a fixed payload type, which causes compatibility issues with clients like VLC. As I showed in the logs above, when I try to connect and play a stream from the RTSP server running on the ESP32-P4, the RTP payload type is always the same — whether I use the MJPEG or H.264 video format — and it's always: m=video 0 RTP/AVP 26 This is problematic because RTSP clients like VLC rely on the payload type described in the SDP, and always interpret it as MJPEG. So, I am not asking to enable external push. I only want more control over the SDP content, particularly the ability to set the correct payload type for the local stream that the RTSP server provides to clients. Or maybe this functionality is already available, and I just need to enable it through some configuration or parameters when creating the server?
@Lextan276 If you want push H264 local can config https://github.com/espressif/esp-adf/blob/master/examples/protocols/esp-rtsp/main/rtsp_service.c#L114
esp_rtsp_video_info_t vcodec_info = {
.vcodec = RTSP_VCODEC_H264,
.width = av_resolution[RTSP_FRAME_SIZE].width,
.height = av_resolution[RTSP_FRAME_SIZE].height,
.fps = VIDEO_FPS,
.len = VIDEO_MAX_SIZE,
};
Currently RTSP server only allow local push, and allow one pull client to connect in. Currently the example code is not suitable for P4. You can use https://components.espressif.com/components/espressif/esp_capture/versions/0.7.0?language=en to easy acquire media data and push use RTSP. Can check https://github.com/espressif/esp-gmf/blob/main/packages/esp_capture/examples/video_capture/main/video_capture.c#L207 for how to do video capture.
@TempoTian , Thank you for the response. Just to clarify: my core issue is not about how to push H.264 locally, but rather about the lack of control over the SDP generated by the RTSP API. I have already confirmed that the RTSP server works fine and outputs video correctly. Currently, the RTSP server automatically generates the SDP with a hardcoded payload type (26 or 96), depending on the codec used. What I am requesting is a way to manually override or customize the SDP — particularly the m= and a=rtpmap lines — so I can define the desired payload type for the codec I’m using. Ideally, the API would allow passing a custom SDP template or at least expose fields for manual modification. Is such functionality currently available or planned?