OBS-studio-webrtc
OBS-studio-webrtc copied to clipboard
CoSMoSoftware
Video stutter using VP8 over WHIP to Janus
Hello,
Using the local network, Janus is reporting lots of "video_retransmissions" coming from OBS. I know you only offer support if the WHIP protocol is being used, so I am making sure to use that protocol. The playback is mostly fine, but using Chrome's "WebRTC internals" shows periods of packet loss (correlating to stutters in playback).
Thanks, Peter
WHIP protocol being used is a necessary condition, but not sufficient. The support is still "best effort". We are all hands on deck for the v26 release. Let me check with lorenzo if his team has some cycles to help you.
No, sorry, quite busy as we are. Anyway, I see Peter already wrote on our forum.
Ah, sorry about the inconvenience, we were not aware of the duplicate post.
@lminiero another question while at it, is your WHIP implementation public?
Not at the moment, no. It wasn't particularly sophisticated, though: it was simply a POC that I used to validate the implementation. I think Peter is using a different WHIP signalling implementation provided by another Janus user.
I am using this implementation provided by Cameron Elliott: https://github.com/x186k/sfu186j.
Thanks.
In any case, if your media is streaming through (however stuttering), it means the signalling / handshake, media path establishment, encryption all went through correctly.
Stuttering can come from bandwidth or CPU limitations, or bad network quality. To differentiate between the former two and the later, you could simply try sending a very low resolution like VGA.
- If you still experience the problem with low rez, your network is likely bad.
- if the low resolution works well, start monitoring your bandwidth and your CPU usage and try with higher resolutions by small increment until you can reproduce the problem again.
HTH.
Are you talking about CPU usage on the encoding side or on the Janus server ingestion side? Both seem to have fairly low usage overall during my tests. I will try to do it at a lower resolution later today, but from memory, I think I did experience stutters at the lower resolution. One thing that did seem to help a little was appending ?pkt_size=1200 to the url.
I can try to run some network tests, but this test is being run locally and both PCs are connected via cat6a (albeit at a gigabit link speed). Also, the external network is a gigabit fiber connection that (from my experience) has never had any packet loss.
What Alex meant was that throughput in general can be impacted, especially since you mentioned very high bitrates that are never seen in webrtc (you talk of 15mbps, normal webrtc apps rarely go beyond 2). It also depends where you're experiencing this stuttering: if in a web page getting video from Janus, then either the browser can't cope with such high bitrate/fps/res, or we simply can't relay that many packets quickly enough.
I am using the Janus VideoRoom demo page to view the stream. I will try again later today, but from my memory I do believe the stuttering was still occurring at around 720p 2 mbps. However, I have even tried at around 1080p 30 mbps 120 FPS and the playback itself is great at times while stuttering at others. Using the Janus recording feature, I looked at the recorded video and the stutter is not present in the recording. Apparently the RTP packets aren’t all arriving in order and within the specified interval (according to someone more knowledgeable than I am on the Google Group). The same stuttering also occurs on multiple web browsers at the same time, leading me to believe the issue is at the encoder or the connection from the encoder to Janus.
Try the sendmmsg branch of Janus too (can't remember the PR number, I'm on my phone), as that's supposed to be more efficient in sending data.
~~@lminiero Alright, I have tried to get the sendmmsg branch working, but I am getting a segmentation fault when running this version while the master version runs without a problem. Do you have any idea what the issue could be?~~
I got it to run by reinstalling libwebsockets. I'll report back with the results of trying to stream using this version.
Using the sendmmsg branch does appear to make things better, at least at lower bitrates and resolutions. However, I am trying to send this video out at at least 120 FPS, 1080p, and at as high of a bitrate as I can.
@agouaillard-cosmo Thanks for the suggestion. Checking the CPU usage on the encoding PC, it is at around 25% usage even though NVENC is supposed to be being used (I can confirm that it is not being used because task manager is showing the GPU is not being used for encoding (maybe a bug?)). On the server PC, the CPU usage is at about 20%. In combination with Lorenzo's suggestion to use a different branch of Janus, it seems like videos at low bitrate and resolution are playing as they should. The issues seem to now occur after around 20000 kbps when using 720p and 120 FPS. Very few frames appear to be dropped at anything under those conditions. Are there any specific settings I should be using on the encoding side for maximum performance? Why is OBS not using NVENC?
My gut feeling is that he available bandwidth is the problem. OBS is using NVENC when sending RTMP. OBS is using vanilla webrtc when sending webrtc, which only uses software codec. As part of a more formal integration of webrtc support in webrtc, there is a design discussion ongoing with the OBS team to unify codec usage.
Bandwidth in terms of network bandwidth? The issues do seem to start to occur predictably at bitrates of around 20000 kbps. However, the network infrastructure here should be more than good enough to support much higher bitrates. I've seen YouTube use 100000 kbps or more when streaming some of the 8k 60 FPS videos.
Could it just be a limitation of WebRTC or maybe a setting on the encoder?
I'm not aware of such limitation. However, I do not have an explicit test about this. When #272 will be done, I will add a dedicated KITE test and we should try again. ETA: a week or two.
Awesome. Thank you!
Also, since I’m using VP8 with software encoding since NVENC is not supported, I am wondering if the software encoder just can’t keep up with those demands. I’m using an i9 9900k and it’s nowhere near maxed out in terms of CPU usage, but my only previous experience with these very high quality streams and videos at bitrates and frame rates that are very high has been using the NVENC encoder on a 2080 Ti (which has easily been able to do things like 1080p 240 FPS at 100000 kbps). I’m really just not sure what the actual problem I’m experiencing is, but it definitely begins right around a specific combination of either 720p 120 FPS 30000 kbps or 1080p 120 FPS 20000 kbps (changing to 60 FPS doesn’t really fix it either). Isn’t it strange to see that kind of behavior where different resolutions affect the maximum bitrate of the stream? If it was some kind of bandwidth bottleneck of the network, wouldn’t both streams experience problems at the same bitrate? That’s why I’m thinking it might have something to do with OBS.
Sounds good. I know this is not a typical use case for WebRTC, but this kind of technology - with its extremely low latency- would be exactly what I desire. I've tried HLS before, and it worked flawlessly with these very high bitrates and frame rates, but the latency was too high. Hopefully we can find a solution (whether the issue lies with OBS or Janus).
Pushed back based on discussion about WHIP draft. Depends now on #301
Is there any update on this? I’m stuck using a much older version that has WHIP support.
Is there any update on this? I’m stuck using a much older version that has WHIP support.
I was thinking the same. Any chance for a custom WHIP endpoint to be added anytime soon to the OBS? The last dev from Lorenzo on Janus side moved the WHIP in front of Janus as a wrapper that can accept http video ingest. Tested it and Lorenzo's whip server works flawlesly (but so far only paired with a whip client based on a gstreamer pipeline). Having the possibility to stream from OBS to a custom WHIP endpoint (which can be the whip server made by Lorenzo would be great. Regards
