jamulus icon indicating copy to clipboard operation
jamulus copied to clipboard

Disable memory swapping on Linux servers

Open dtinth opened this issue 1 year ago • 4 comments

Short description of changes

Updated systemd unit file to disable swap on Jamulus process.

CHANGELOG: Server: Disabled swap memory usage on Linux.

Context: Fixes an issue?

I noticed that Jamulus server randomly (intermittently) become unresponsive for 0.3 – 1 seconds every once in a while. (This issue has happened for a long time prior.) This caused everyone to experience some sound drops (we call it "ตกหลุมอากาศ," an aviation term that refers to the experience of a cruising plane hitting an air pocket causing turbulence).

Last month I installed Netdata on my servers to collect detailed metrics. It seems that Linux has been paging some memory to/from disk every once in a while around the unresponsiveness occurs.

So systemd has the ability to prevent an app from using swap space. This might help with this issue. From my preliminary testing, it seems to have helped, albeit not completely.

As memory footprint of Jamulus server is very minimal (19.7M used after 8 months of uptime) I think this config change should not have any negative impact unless the server is run on a potato.

Does this change need documentation? What needs to be documented and how?

No.

Status of this Pull Request

Ready

What is missing until this pull request can be merged?

N/A

Checklist

  • [x] I've verified that this Pull Request follows the general code principles
  • [x] I tested my code and it does what I want
  • [x] My code follows the style guide
  • [x] I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
  • [x] I've filled all the content above

dtinth avatar Sep 24 '24 13:09 dtinth

Thanks. But why would the OS swap in the first place? The app is active and ram should be available for such a small footprint.

Maybe the server is running too much other stuff?

ann0see avatar Sep 24 '24 20:09 ann0see

Yeah, the server runs other stuff too (such as a system that broadcasts MP3 streams, some web servers), so that’s a possible cause. I did cat /proc/<pid>/smaps before the change and some memory is indeed in the swap area.

The system I was running happens to have vm.swappiness = 60 (default value), so I think there is always a chance that the system moves the memory to swap area despite Jamulus having a low memory footprint and the system having plenty of free RAM.

It may be worth noting that many cloud servers use a network-attached disk, so read/writes to disk becomes network I/O.

dtinth avatar Sep 24 '24 21:09 dtinth

I've added this change to my own servers, but I can't actually test properly until I'm back home after my holiday.

softins avatar Sep 26 '24 12:09 softins

Added to my servers and directories. I've a few other bits:

[Service]
CPUSchedulingPolicy=fifo
CPUSchedulingPriority=20
IOSchedulingClass=realtime
IOSchedulingPriority=3
MemorySwapMax=0

No idea if the differences help.

pljones avatar Oct 12 '24 15:10 pljones

@softins do you think this is ok?

ann0see avatar Dec 04 '24 22:12 ann0see

@softins do you think this is ok?

I added it to my server on AWS some time ago. It didn't make anything worse, but it was hard to tell if it made any improvement.

I've since moved my server to Vultr, and forgot to transfer that setting. I've just added it back and restarted the jamulus services, so we'll see. I've just discovered that top has a SWAP column that can be added to the display, to show the swap usage of each process.

I'm on Debian 12, which has systemd 252. I've read that this setting is only available in systemd 239 and newer, but I don't know what versions older distros use.

I've no objection to merging this.

softins avatar Dec 04 '24 22:12 softins

Ok. Let's merge this. I don't think there are any disadvantages unless it's run on low RAM machines.

ann0see avatar Dec 05 '24 21:12 ann0see