earlyoom
earlyoom copied to clipboard
rfjakob
can not identify process.
Nov 05 10:51:12 hybrid01 earlyoom[1105]: mem avail: 9317 of 64154 MiB (14.52%), swap free: 0 of 0 MiB ( 0.00%) Nov 05 11:34:49 hybrid01 earlyoom[1105]: mem avail: 1259 of 64154 MiB ( 1.96%), swap free: 0 of 0 MiB ( 0.00%) Nov 05 11:34:49 hybrid01 earlyoom[1105]: low memory! at or below SIGKILL limits: mem 2.00%, swap 100.00% Nov 05 11:34:49 hybrid01 earlyoom[1105]: sending SIGKILL to process 3887 uid 1009 "java": badness 67, VmRSS 4352 MiB
There are many java process in one machine. I can not identify "earlyoom" killed which one. Can you print all process information ?
I can suggest nohang as an alternative. It prints cgroup, cmdline etc (it's configurable). For example:
ноя 17 19:15:25 PC nohang[3824]: Memory status that requires corrective actions:
ноя 17 19:15:25 PC nohang[3824]: MemAvailable [47 MiB, 0.4 %] <= soft_threshold_min_mem [590 MiB, 5.0 %]
ноя 17 19:15:25 PC nohang[3824]: SwapFree [4587 MiB, 9.7 %] <= soft_threshold_min_swap [4720 MiB, 10.0 %]
ноя 17 19:15:25 PC nohang[3824]: Found 97 tasks with non-zero oom_score (except init and self) in 8ms
ноя 17 19:15:25 PC nohang[3824]: TOP-15 tasks by badness:
ноя 17 19:15:25 PC nohang[3824]: Name PID badness
ноя 17 19:15:25 PC nohang[3824]: --------------- ------- -------
ноя 17 19:15:25 PC nohang[3824]: python3 3829 1233
ноя 17 19:15:25 PC nohang[3824]: firefox-esr 2005 676
ноя 17 19:15:25 PC nohang[3824]: Web Content 2140 669
ноя 17 19:15:25 PC nohang[3824]: Web Content 2147 669
ноя 17 19:15:25 PC nohang[3824]: Web Content 2161 668
ноя 17 19:15:25 PC nohang[3824]: Privileged Cont 2239 668
ноя 17 19:15:25 PC nohang[3824]: WebExtensions 2321 668
ноя 17 19:15:25 PC nohang[3824]: Web Content 2418 668
ноя 17 19:15:25 PC nohang[3824]: Web Content 3293 668
ноя 17 19:15:25 PC nohang[3824]: Web Content 3326 668
ноя 17 19:15:25 PC nohang[3824]: Web Content 3564 668
ноя 17 19:15:25 PC nohang[3824]: Web Content 3604 668
ноя 17 19:15:25 PC nohang[3824]: kate 9555 667
ноя 17 19:15:25 PC nohang[3824]: systemd-journal 382 666
ноя 17 19:15:25 PC nohang[3824]: systemd-timesyn 768 666
ноя 17 19:15:25 PC nohang[3824]: TOP printed in 0ms; process with highest badness:
ноя 17 19:15:25 PC nohang[3824]: PID: 3829, name: python3, badness: 1233
ноя 17 19:15:25 PC nohang[3824]: Recheck memory levels...
ноя 17 19:15:25 PC nohang[3824]: Memory status that requires corrective actions:
ноя 17 19:15:25 PC nohang[3824]: MemAvailable [46 MiB, 0.4 %] <= soft_threshold_min_mem [590 MiB, 5.0 %]
ноя 17 19:15:25 PC nohang[3824]: SwapFree [4572 MiB, 9.7 %] <= soft_threshold_min_swap [4720 MiB, 10.0 %]
ноя 17 19:15:25 PC nohang[3824]: Victim status (found in 1ms):
ноя 17 19:15:25 PC nohang[3824]: PID: 3829, name: python3, state: R (running), EUID: 1000, SID: 3812 (bash), lifetime: 31.9s
ноя 17 19:15:25 PC nohang[3824]: badness: 1233, oom_score: 1233, oom_score_adj: 0
ноя 17 19:15:25 PC nohang[3824]: Vm, MiB: Size: 55137, RSS: 9341 (Anon: 9336, File: 5, Shmem: 0), Swap: 40752
ноя 17 19:15:25 PC nohang[3824]: cgroup_v1:
ноя 17 19:15:25 PC nohang[3824]: cgroup_v2: /user.slice/user-1000.slice/session-4.scope
ноя 17 19:15:25 PC nohang[3824]: ancestry: PID 3812 (bash) <= PID 3705 (lxterminal) <= PID 1252 (mate-panel)
ноя 17 19:15:25 PC nohang[3824]: exe realpath: /usr/local/bin/python3.8
ноя 17 19:15:25 PC nohang[3824]: cwd realpath: /home/user
ноя 17 19:15:25 PC nohang[3824]: Memory info, MiB:
ноя 17 19:15:25 PC nohang[3824]: total=11799, used=10748, free=251, available=44, shared=29, buffers=171, cache=629,
ноя 17 19:15:25 PC nohang[3824]: swap_total=47195, swap_used=42624, swap_free=4571
ноя 17 19:15:25 PC nohang[3824]: Memory pressure (system-wide):
ноя 17 19:15:25 PC nohang[3824]: some avg10=36.37 avg60=15.17 avg300=6.08
ноя 17 19:15:25 PC nohang[3824]: full avg10=36.37 avg60=15.16 avg300=6.08
ноя 17 19:15:25 PC nohang[3824]: Implementing a corrective action:
ноя 17 19:15:26 PC nohang[3824]: Sending SIGTERM to the victim
ноя 17 19:15:26 PC nohang[3824]: OK; total response time: 10ms
ноя 17 19:15:26 PC nohang[3824]: What happened in the last 34.2s:
ноя 17 19:15:26 PC nohang[3824]: [ OK ] Sending SIGTERM to python3: 1
ноя 17 19:15:26 PC nohang[3824]: The victim died in 4.618s
ноя 17 19:15:26 PC nohang[3824]: Memory status after implementing a corrective action:
ноя 17 19:15:26 PC nohang[3824]: MemAvailable: 9506.4 MiB, SwapFree: 45304.8 MiB
