[BUG] OBS Beam Plugin: Frame Drops, Audio-Video Desync, and Buffer Issues Under High Load or System Interruptions

[kuskryptus]

Bug Report:

Summary

We are experiencing glitches with the OBS Beam plugin when sending uncompressed video over an Ethernet connection. Despite using high-performance laptops for both sending and receiving, the plugin struggles with frame buffering, resulting in dropped frames and warnings. Additionally, there is a significant issue with audio-video desynchronization when the system undergoes interruptions or changes in load, such as restarting the sending laptop or connecting a new HDMI display.

Interestingly, the plugin works without glitches and errors when using a clean OBS setup with minimal plugins, filters, and inputs. However, when OBS is configured for real production use, including plugins, HDMI cameras, and multiple inputs and filters, the video glitches and buffering issues become prominent. The problem is difficult to reproduce with a clean OBS installation.

The main focus is to make the OBS Beam plugin more resistant to high loads and sudden changes in system conditions, such as when CPU or GPU usage spikes or when new HDMI cameras are connected. These situations often cause the system to freeze momentarily, which in turn leads to desynchronization and other performance issues with the plugin.

System Specifications

Both the OBS Beam plugin 1.1.0 and OBS Studio (v31.0.0) are updated to their latest versions at the time of this report.

Receiving Laptop:

• Model: High-end laptop with NVIDIA GeForce RTX 4060 Laptop GPU
• CPU: 12th Gen Intel(R) Core(TM) i7-12700H @ 2.68 GHz
• Memory: 16 GB (6.7 GB free during testing)
• Operating System: Windows 11 Pro (64-bit), version 23H2, build 22631
• OBS Version: 31.0.0 (64-bit, portable mode)
• Additional Details:
  • Game DVR: Enabled
  • Game Mode: Enabled
  • Browser Hardware Acceleration: Enabled

Sending Laptop:

• Model: High-end laptop with NVIDIA GeForce RTX 4070 Laptop GPU
• CPU: 13th Gen Intel(R) Core(TM) i7-13650HX @ 2.80 GHz
• Memory: 32 GB (26.7 GB free during testing)
• Operating System: Windows 11 Pro (64-bit), version 24H2, build 26100
• OBS Version: 31.0.0 (64-bit, portable mode)
• Additional Details:
  • Game DVR: Disabled
  • Game Mode: Enabled
  • Browser Hardware Acceleration: Enabled

Observed Behavior

1. Frame Buffer Issues:

   • Receiving laptop logs frequently show warnings like "Frame buffer below target" and "Missing video frame," with timestamps indicating significant frame delays.
   • Example logs:

     16:07:56.468: [xObsBeam] Warning: Frame buffer below target: 112/120
     16:07:56.534: [xObsBeam] Warning: Missing video frame 549196869863 in frame buffer, timestamp deviation of 333333330 > max deviation of 33666666, consider increasing the frame buffer time if this happens frequently.
     

2. Send Queue Overload:

   • Sending laptop logs indicate issues with the send queue size, suggesting overload despite low system resource usage.
   • Example logs:

     16:24:21.248: [xObsBeam] <10.77.9.204:55906> Warning: Send queue size 7 (15) at video frame 1599266549988.
     

3. Audio-Video Desynchronization:

   • When the buffer is set to 1000ms, any interruption or change in system load (e.g., restarting the sending PC or connecting a new HDMI display) causes frequent desync issues. The audio and video streams lose synchronization and do not recover automatically.
   • This issue occurs even when hardware resources and network bandwidth are not under stress.

4. Performance Discrepancy:

   • Both laptops show minimal CPU and GPU load during testing, suggesting the issue is not hardware-related.
   • OBS Beam works well in a clean OBS setup but struggles in real production environments with multiple plugins and inputs.

Steps to Reproduce

1. Connect the sending and receiving laptops via Ethernet.
2. Configure OBS Beam plugin to transmit uncompressed video at 1920x1080, 30 fps.
3. Start streaming video from the sending laptop to the receiving laptop.
4. Add multiple plugins, inputs, and filters to the OBS setup.
5. Restart the sending PC or connect/disconnect an external HDMI display.
6. Observe the logs on both devices for warnings and errors, and monitor for audio-video desynchronization.

Expected Behavior

The OBS Beam plugin should transmit uncompressed video without frame drops, buffering issues, or audio-video desynchronization, even in real production environments with a fully loaded OBS setup.

Actual Behavior

Frequent frame drops, buffer underruns, and audio-video desynchronization occur when OBS is configured for real production use. Increasing the frame buffer time did not resolve the issue.

Logs

Receiving Laptop Logs (Excerpts):

16:07:56.468: [xObsBeam] Warning: Frame buffer below target: 112/120
16:07:56.501: [xObsBeam] Warning: Missing video frame 549196869863 in frame buffer, timestamp deviation of 333333330 > max deviation of 33666666, consider increasing the frame buffer time if this happens frequently.

Sending Laptop Logs (Excerpts):

16:24:21.248: [xObsBeam] <10.77.9.204:55906> Warning: Send queue size 7 (15) at video frame 1599266549988.

Potential Cause

The issue might be related to:

1. Inefficient buffering in the OBS Beam plugin.

2. Poor handling of system interruptions or load changes by the plugin.

3. Plugin performance degradation in complex OBS setups.

Troubleshooting Steps Taken

• Increased frame buffer time and render delay limit on receiving laptop.
• Verified hardware and network performance (no significant bottlenecks observed).
• Set up Windows on both laptops for the best possible performance by disabling unnecessary services and optimizing system settings.
• Simplified OBS setups to confirm the plugin works without glitches in clean installations.

[YorVeX]

If this is a gigabit Ethernet connection, the ~700 Mbps Beam would need for 1080p30 (assuming default NV12 setting in OBS) should have enough headroom, but not if you have a lot of other things going through that same network. Are you running anything else when your full production setup is up that might need significant chunks of bandwidth, at least from time to time? Especially something maybe wasn't on when comparing to the more simple setup you didn't have issues with?

The one thing that is quite specific to Beam is that it sends very large packets compared to the usual traffic you have on a typical network, and many network card driver settings are by default not optimized for that, which might not be an issue for 99% of the things you're doing with that network, but for Beam. Just to rule that out, can you please go to your network driver settings on both sender and receiver laptop and try to set any setting that says something with buffer, especially like "transmit buffer", "send buffer", "receive buffer" to their maximum values - just enter a very high number like 99999 for each of them and then it should tell you what the actual max is, then you pick that. This seems to give a few good examples of what to look for.

If you have a chance, you could also try to run iPerf to find out the actual speed you're getting over that network interface, unfortunately it's not super simple to use, but also not too hard and there are a few tutorials online.

Since you brought optimization in relation to OBS performance up, there is one setting in Beam that has a strong impact on that, choosing between different trade-offs. On the sender side, you might want to try and disable the "Recommended options only" setting at the top, which makes this option visible:

See if changing this makes any difference about the issue in your production setup.

Another thing worth testing is enabling the Density compression at the lowest strength of 1. Even with full compression level of 10, in my tests I found scenarios where not only the load stayed the same with this on compared to raw, but where the load was even a bit lower. My guess is that this is because pushing huge amounts of data is also putting strain on the CPU, especially on not well optimized consumer 1 gig Ethernet chips, and if that bandwidth is reduced by a bit with minimal CPU work (Density is extremely efficient), this might balance out each other. Since it's both reducing the bandwidth requirement by a bit and at least changing the CPU load profile a bit, it definitely could help here. And since it's a lossless algorithm, you don't lose any quality.

[MarekPro1]

My try on this you didn’t get right the issue.

Description: When using the Beam plugin with OBS, the plugin unexpectedly changes its internal delay, leading to audio and video desynchronization. Initially, synchronization is achieved by adjusting the render buffer, but Beam later modifies its delay when there are any hiccups on the sender side, such as unplugging the monitor or during network stress tests. The issue can be temporarily resolved by reopening the source on the receiver side and clicking “OK” to reset the Beam connection.

Steps to Reproduce: 1. Setup: • Sender Side: • Network Card: 10 Gbit • Monitor: Connected • Jumbo Packets: Enabled • Receiver Side: • Network Card: 2.5 Gbit • OBS with Beam Plugin installed • Jumbo Packets: Enabled 2. Configuration: • Set the render buffer on the receiver to 0ms, 1000ms, or 4000ms. 3. Procedure: • Start streaming or recording with OBS using the Beam plugin. • Manually synchronize audio and video within OBS. 4. Trigger the Issue: • Cause any hiccup on the sender side, such as: • Unplugging the monitor. • Initiating a network stress test. • Any other interruptions or fluctuations in the sender’s system/network. 5. Temporary Fix: • On the receiver side, open the Beam source and click “OK” to reset the connection.

Expected Behavior: Beam maintains a consistent internal delay, keeping audio and video streams synchronized without manual intervention, even when there are minor disruptions on the sender side.

Actual Behavior: Beam alters its internal delay after any hiccup on the sender side, causing A/V desynchronization. Resetting the Beam connection temporarily restores synchronization.

Notes for Developers: This issue affects the reliability of streaming setups that rely on Beam for video transmission while handling audio directly through OBS. Ensuring that Beam maintains a stable internal delay, even during minor disruptions on the sender side, is crucial for consistent audio-video synchronization.

[YorVeX]

Without buffering (meaning 0 ms), you will always have a change in delays, slowly over time, and also as single events from such lags - that's coming from how OBS works and is expected. It's the reason I added the buffering feature in Beam, to have something to counter this. 1000 ms is a bit close, but anything at 1500 ms or above should give Beam enough headroom to take its countermeasures against the normal delay fluctuations that can occur even when everything is otherwise smooth. If you have a specific event that causes a short visible lag or freeze in OBS, then the buffer might need to be even higher. 4000 ms would be a good pick and indeed should be enough if you can live with that kind of delay.

Since you seem to have a way of reliably being able to reproduce such a lag inducing event causing your desync issue, can you try please to run the receiver OBS with the --verbose launch parameter and check the log while the buffer changes its delay? Beam will log what delay it detects and each step of adjusting to compensate changes in OBS delay. Will be very helpful to see which part of the calculation goes wrong.

Something you can already check without using verbose logs: what does the render delay in the Beam Receiver properties say at the beginning when everything is still OK compared to after the issue occurred? Need to press the "Refresh delay info" button to update it:

It should always stay within the range of what you set, with a bit of variance, like in my example where I set 1500 and get 1541 ms actual delay.

Manually synchronize audio and video within OBS

How do you do this? Note that the issue that OBS can change delays of various sources and filters can affect any source and any filter in OBS, especially sources that use any sort of delay filter can have their delay change. Beam is pretty much the only source that is safe from this, and not actually safe, it just counters it with the buffer (while any other source doesn't). And I think even just using only the audio offset can be affected by this. So make sure that you don't use any render/video delay filters and audio offsets in OBS, they can and will change their delays and then it no longer matches the delay of the video feed.

Also, how do you send audio? Can you validate that it's not the audio feed changing its delay? And where do you get the desync, in the final stream or final recording or in the monitoring output on headphones or speakers? Note that audio monitoring output has its own desync issue in OBS in addition to all the other issues.

[MarekPro1]

Yes we found out that using buffer 4000ms it helps as like you said now.

Thats very cool to use --verbose, we will use it for also many other things that we are debugging.

Okay refresh delay info is new for us, i don't get why we didn't saw it.

We know about all of obs, weak synchronization, thats why we only delay video in beam and audio in DAW (Ableton live) audio gets send thrue VB Matrix, that creates asio device and that asio device is grabbed in obs by ASIO plugin. And for checking sync we are using this very clever plugin https://obsproject.com/forum/resources/audio-video-sync-dock.2028/

okay we will try to check the delay info in beam, but yesterday we were testing it whole day trying to desynchronize, by unpluging monitors, network cards, chargers, stress test runned. The only thing that it desynchronized was we unpluged the Usb-C 2.5Gbit network card at reciever side and only then it shifted 40ms, but when in beam we just clicked OK in source it returned.

Soo from yesterday testing it seemed pretty robust (but it was on our second live concerts rig).

[YorVeX]

Any update on this? Has it been stable with a high enough buffer?

[MarekPro1]

Hello, with buffer 4000ms we did not had any issue from the last time we spoke. But we have another streaming station and on that we have only 1500ms buffer and here sometimes happen that we need to click OK in reciever source and it synces.

[zbynekdrlik]

Up again with this as we have delay glitches with 4000ms buffer also and reset beam fix that descync.