SoC ideas (especially esp32-cam)

[tugrul]

Hi,

First, thank you for this library. API is pretty good and it has less footprint on build.

I targeted to build a working sample for esp32-cam. I encountered to major limitations.

First OpenSSL is huge for this kind of devices. mbedTLS or WolfSSL are good alternatives and ESP-IDF toolchain's documentation says https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/protocols/openssl_apis.html

The code of this API (located in openssl directory), does not contain OpenSSL itself but is intended as a wrapper for applications using the OpenSSL API. It uses mbedTLS to do the actual work, so anyone compiling openssl code needs the mbedtls library and header file.

Maybe there is no necessary changes required and wrapper can compensate OpenSSL requirements.

Second limitation is cpu power is limited and not eligible for h264 encoding or etc. There is a one thing. esp32-camera library can supply jpeg image frames and RTP protocol has support jpeg streaming (mjpeg) by RFCs. https://datatracker.ietf.org/doc/html/rfc2435 https://datatracker.ietf.org/doc/html/rfc2035

Also I see some code on chromium's webrtc repository looks implemented mjpeg feature. https://chromium.googlesource.com/external/webrtc/src/+/refs/heads/master/common_video/jpeg/

I'm not sure is actually mjpeg feature exists on latest builds of web browsers because there is no information on latest documentation for this.

Thanks, Tuğrul

[paullouisageneau]

First OpenSSL is huge for this kind of devices. mbedTLS or WolfSSL are good alternatives and ESP-IDF toolchain's documentation says https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/protocols/openssl_apis.html

It seems the wrapper lacks the DTLS functions from OpenSSL, so it can't be a drop-in replacement. However, it would be possible to write another DTLS backend for mbedTLS like already implemented for GnuTLS and OpenSSL.

WolfSSL has an OpenSSL wrapper, but it's licensed under GPL, not LGPL, which would require changing the license of libdatachannel to GPL when compiled against WolfSSL.

Second limitation is cpu power is limited and not eligible for h264 encoding or etc. There is a one thing. esp32-camera library can supply jpeg image frames and RTP protocol has support jpeg streaming (mjpeg) by RFCs. https://datatracker.ietf.org/doc/html/rfc2435 https://datatracker.ietf.org/doc/html/rfc2035

Also I see some code on chromium's webrtc repository looks implemented mjpeg feature. https://chromium.googlesource.com/external/webrtc/src/+/refs/heads/master/common_video/jpeg/

I'm not sure about actual support. Sadly, I don't think a browser will accept to negotiate mjpeg as it is not supposed to be supported in WebRTC for streaming, and I think the code here is only to decode frames for webcams that don't support a better video encoding method.