Camera fails to start streaming

[djm181]

Bug Report

Describe the Bug

I have a handful of cameras that will never start a stream with API v11. They worked fine on v9. I updated to 11.0.2 today since I'd been watching #930 but unfortunately this didn't resolve the issue.

To Reproduce

Steps to reproduce the behavior:

• Create a RestApi
• Attempt to start a stream on this specific camera using camera.streamVideo
• The stream appears to start, in that the streamVideo promise resolves, but upon inspecting the HLS stream, segments are not being written (as you can see from the logs)

Expected behavior

Stream starts

Screenshots/Logs

stream_fail.csv

Additional context

I have a bunch of other cameras that start without issue.

Environment

• OS: Alpine linux
• Node.js: 16.14.2
• ring-client-api: 11.0.2

[djm181]

Interestingly, if I add -map 0:v to my ffmpeg options, so that the audio stream is dropped, then video streaming works perfectly (albeit without sound). This made me wonder if it was something odd in the audio for this camera, but the SDP looks normal for Opus to me:

  ring v=0
  ring o=- 8845816573485370895 0 IN IP4 0.0.0.0
  ring s=rms4d7c3277
  ring t=0 0
  ring a=ice-options:trickle
  ring m=audio 9 UDP/TLS/RTP/SAVPF 101
  ring c=IN IP4 0.0.0.0
  ring a=setup:actpass
  ring a=rtcp-mux
  ring a=rtcp-rsize
  ring a=sendrecv
  ring a=rtpmap:101 OPUS/48000/2
  ring a=rtcp-fb:101 nack pli
  ring a=fmtp:101 useinbandfec=1;sprop-stereo=0
  ring a=ssrc:830651158 msid:user3352565587@host-960ffa5c webrtctransceiver1376
  ring a=ssrc:830651158 cname:user3352565587@host-960ffa5c
  ring a=mid:audio0
  ring m=video 9 UDP/TLS/RTP/SAVPF 96 97
  ring c=IN IP4 0.0.0.0
  ring a=setup:actpass
  ring a=rtcp-mux
  ring a=rtcp-rsize
  ring a=sendrecv
  ring a=rtpmap:96 H264/90000
  ring a=rtcp-fb:96 nack
  ring a=rtcp-fb:96 nack pli
  ring a=fmtp:96 packetization-mode=1;profile-level-id=640029;level-asymmetry-allowed=1
  ring a=rtpmap:97 rtx/90000
  ring a=fmtp:97 apt=96
  ring a=ssrc-group:FID 75086872 770031604
  ring a=ssrc:75086872 msid:user3352565587@host-960ffa5c webrtctransceiver1377
  ring a=ssrc:75086872 cname:user3352565587@host-960ffa5c
  ring a=ssrc:770031604 msid:user3352565587@host-960ffa5c webrtctransceiver1377
  ring a=ssrc:770031604 cname:user3352565587@host-960ffa5c
  ring a=mid:video1

[dgreif]

The big change from 11.0.0 to 11.0.2 was to handle cameras that only support PCMU (not opus). I should be handling the intricacies here, but it could have a bug somehow. Are you using the startTranscoding in your implementation? Could you add some logging to see if it is indeed using pcmu on the cameras that fail?

[djm181]

The SDP that I posted in my last comment was dumped directly from the onCallAnswered handler in StreamingSession.bindToConnection - so it does appear the camera is sending opus audio. Is there anything else I could log to help diagnose this?

[dgreif]

Is there anything else the cameras have in common - Ring Edge, model, etc?

And just to clarify, is the webrtc connection getting to connected, and then it's just ffmpeg that is failing to properly consumer the stream?

[djm181]

No similarity in camera models - they are stickup_cam_lunar, stickup_cam_v4 and floodlight_v2, but at the same time I have other cameras on the same models (except the floodlight, that one's unique) that stream properly.

And yes, that is accurate - the webrtc connection is working, it's just ffmpeg. Happy to do a debugging session or chat on discord if you like. I'm able to reproduce on my machine using the browser-example code.

[dgreif]

Thanks for the details. I'm wondering if it's simply something with the ffmpeg setup. Could you try the record-example instead of browser example to see if it has issues with the same cameras?

[dgreif]

Let me know if 11.0.3 is any better. It has a number of bug fixes and an updated version of ffmpeg

[djm181]

record example has the same issue. In 11.0.2 it ends prematurely with a Conversion failed! message. In 11.0.3 after making sure the dependencies are updated it hangs indefinitely. And if I add the -map option to drop audio then it works again.

[dgreif]

I haven't seen any other reports of this issue so far. I think the only way for me to debug further would be to test with one of the cameras that are causing issues. Are any of them in a spot that you would feel comfortable sharing it with me? If so, you can share it with [email protected] and I'll take a look when I get a chance. If not, I think the debugging will need to happen on your end.

[djm181]

Thanks Dusty - I have shared one of the cameras with you. Let me know if there is any way I can help here.

[dgreif]

Well I was able to recreate the issue. From what I can tell, there just aren't any audio packets flowing. I was also not able to hear any audio through the web or ios Ring app, but it might just be that Ring is doing advanced noise cancelling. I don't have much time to look into it more this week, but maybe you could try making some noise in front of the camera to verify you can hear it through the Ring app, and see if making noise gets ffmpeg unstuck

[tsightler]

Hey @dgreif. I'm curious if you ever made any progress on this issue? I think I may have a few users reporting something similar. It doesn't seem to be widespread, but sometimes a user will have one camera that just will not seem to start a stream as the stream seems to just hang. I've only had a handful of such reports, and for some users, the cameras will, very occassionally, work, while all of their other cameras work fine. I can't reproduce on my cameras, but they are all pretty old now and thus all have some kind of noise all the time. It may not be the same issue, but I thought I'd check and see if you've had any success tracking this down.

[dgreif]

@tsightler I have not made any more progress on this. Seems like an ffmpeg config issue, but I'm not sure what's causing it

[tsightler]

Thanks for the update @dgreif, if I ever get any access to a camera where I can reproduce the behavior, I'll dig into it myself.

[tsightler]

Hi @dgreif, I've had a couple of ring-mqtt users share their cameras with me so that I could dig a little into the behavior and, at least for the cases I've been able to investigate, it seems the problems are more about how long the cameras take to start streaming than any real bug with ring-client-api or ring-mqtt itself, so I would suggest that you ignore my comments on this issue.

I'm not sure why certainly cameras take 3-5 seconds longer than others to start the WebRTC stream, while others seems to do so in <2 seconds, but, because I'm using RTSP, and most RTSP clients time out in 5 seconds, cameras that take longer than a few seconds to start the stream will not work with ring-mqtt, but that's not a bug in ring-client-api.

[dgreif]

Thanks for the update @tsightler. I think delayed streaming is probably just a symptom of high latency networks going round trip to the ring servers, and not something we can really solve. Hopefully you can find a way to handle the delay on your end, maybe by starting RTSP later.

@djm181 sorry I haven't been able to find a solution for your original issue with audio packets. I haven't seen this cause any issues on the homebridge side, so I think I'm going to have to leave this for you to track down if it's still causing issues.