earlyoom icon indicating copy to clipboard operation
earlyoom copied to clipboard

earlyoom kills more than required when ram isn't cleaned fast enough

Open 9SMTM6 opened this issue 6 months ago • 7 comments

This is probably connected to https://github.com/rfjakob/earlyoom/issues/284, and not reliably repeatable. But when running earlyoom to test the usecase that made me research tools such as it, I found it to occasionally kill a few processes more than should be required.

Here is a log of it happening (at about 14:00) and not happening (at about 14:45):

Feb 16 13:58:29 cloakOfManyColors earlyoom[1233]: mem avail:  2430 of 31808 MiB ( 7.64%), swap free: 1419 of 15903 MiB ( 8.93%)
Feb 16 13:58:29 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 13:58:29 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 29410 uid 1000 "cluster": badness 1264, VmRSS 23964 MiB
Feb 16 13:58:29 cloakOfManyColors earlyoom[1233]: process exited after 0.1 seconds
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: mem avail:  2511 of 31808 MiB ( 7.90%), swap free: 1317 of 15903 MiB ( 8.28%)
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 11567 uid 1000 "electron": badness 869, VmRSS 17 MiB
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: process exited after 0.1 seconds
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: mem avail:  2568 of 31808 MiB ( 8.08%), swap free: 1424 of 15903 MiB ( 8.96%)
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 11239 uid 1000 "Web Content": badness 822, VmRSS 14 MiB
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: process exited after 0.1 seconds
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: mem avail:  2651 of 31808 MiB ( 8.34%), swap free: 1349 of 15903 MiB ( 8.49%)
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 11094 uid 1000 "Web Content": badness 822, VmRSS 14 MiB
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: process exited after 0.1 seconds
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: mem avail:  2741 of 31808 MiB ( 8.62%), swap free: 1273 of 15903 MiB ( 8.01%)
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 11322 uid 1000 "Web Content": badness 822, VmRSS 14 MiB
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: process exited after 0.1 seconds
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: mem avail:  2818 of 31808 MiB ( 8.86%), swap free: 1198 of 15903 MiB ( 7.54%)
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 11781 uid 1000 "rust-analyzer": badness 812, VmRSS 3 MiB
Feb 16 13:58:30 cloakOfManyColors earlyoom[1233]: process exited after 0.0 seconds
Feb 16 14:19:39 cloakOfManyColors earlyoom[1233]: mem avail: 24073 of 31808 MiB (75.68%), swap free: 13628 of 15903 MiB (85.69%)
Feb 16 14:46:48 cloakOfManyColors earlyoom[1233]: mem avail:     0 of 31808 MiB ( 0.00%), swap free: 1571 of 15903 MiB ( 9.88%)
Feb 16 14:46:48 cloakOfManyColors earlyoom[1233]: low memory! at or below SIGTERM limits: mem 10.00%, swap 10.00%
Feb 16 14:46:48 cloakOfManyColors earlyoom[1233]: sending SIGTERM to process 31647 uid 1000 "cluster": badness 1293, VmRSS 26345 MiB
Feb 16 14:46:49 cloakOfManyColors earlyoom[1233]: process exited after 0.1 seconds

interestingly they are connected: these processes are from vscode and rust-analyzer (lsp for rust, was running for the killed cluster process). And it wasn't fatal, vscode recovered after a warning about the kill.

I'm not sure where their high badness score comes from, and also not why the memory wasn't freed immediately after killing the root cause. The VmRSS noted in the logs is miniscule and when observing the processes separately they don't take up enough memory to be problematic during the spike.

This was done on kernel 6.7.4-arch1-1, if its helpful.

Feel free to close this. If there are no questions from you, and no problematic situations happening to me in the future, I will not put in more time on my own. I just wanted to report this for completeness.

9SMTM6 avatar Feb 16 '24 14:02 9SMTM6