iOS SDK stops sending stream after 30 seconds when the camera captures black or static things

[mekya]

iOS sample app stops sending stream after 30 seconds when the camera captures black or static things

Steps to reproduce

1. Make the target resolution to 1920x1080 -
2. Set the bitrate to 2000*1000
3. Publish stream to Ant Media Server
4. Watch the stream on Ant Media Server
5. Put device on the table so that camera captures just black view
6. Wait for 30 seconds

Expected behavior

iOS SDK should continue sending data to the Ant Media Server. In other words, after camera starts to capture somethings else, we should watch the stream on Ant Media Server side.

Actual behavior

The connection drops on the server side and iOS does not send any stream data to the server.

[yashtandon113]

We've found the problem. iOS H264 Encoder does not support over 1440x810 resolution with WebRTC encoding parameters (Real-time & Baseline 3.1). ​ We've compiled the WebRTC framework in debug mode to understand why it stops sending video frames. It turns out that iOS devices sends stream with lower resolutions at first and then it increases over time. In other words, when we set target resolution to 1920x1080, WebRTC starts to send frames with 360p and it increases over time according to the network conditions. When it increases over 1440x810, H264 Encoder gives the following error

(RTCVideoEncoderH264.mm:780): H264 encode failed with code: -12902

Then the question is this why are you having this problem when the camera view is black? The answer is that when the camera view is black, the bitrate of the video decreases because it's just a black screen. Then WebRTC thinks that there is an opportunity to increase the resolution and it increases resolution to over 1440x810. This is also why we don't have this problem when we set the target resolution to 1280x720.

iOS supports full HD video recording but it's not the same settings with WebRTC encoder settings. ​ The quick workaround solution is to keep the resolution 1280x720. ​ The permanent solution is that we can try to change the profile than Baseline 3.1 and check that if it works. Changing baseline may have some consequences in the playback side as well. So we're working these internal/deep-level things for WebRTC.