Restart cycle freezes debian

[locint]

I experienced something today and not for the first time but this time I dedided to look into system logs.

The restart cycle somehow manages to cause my debian home server to freeze. I've been running a cluster of three servers without any issues.

Memory usage is around 30GB/64GB but apparently I ran out of memory.

CPU: 5600g

So far I can tell that there was a lot of restart spam in the logs as well for serveral hours.

Sep 18 14:18:58 debian-homeserver kernel: [351120.866164] [1717124] 1000 1717124 1062 635 53248 0 0 restart_server.

Snapshot of the logs moments before the system freezes:

Sep 18 15:41:15 debian-homeserver kernel: [356057.516154] [2170526] 1000 2170526 1062 146 45056 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516158] [2170527] 1000 2170527 1 1 12288 0 0 restart_server. Sep 18 15:41:15 debian-homeserver kernel: [356057.516162] [2170528] 1000 2170528 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516167] [2170529] 1000 2170529 1062 146 49152 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516171] [2170530] 1000 2170530 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516175] [2170531] 1000 2170531 1062 146 53248 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516179] [2170532] 1000 2170532 1062 147 45056 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516183] [2170533] 1000 2170533 1062 147 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516187] [2170534] 1000 2170534 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516191] [2170535] 1000 2170535 1062 148 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516195] [2170536] 1000 2170536 1 1 12288 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516197] [2170537] 1000 2170537 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516200] [2170538] 1000 2170538 1062 148 49152 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516202] [2170539] 1000 2170539 1062 147 49152 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516204] [2170540] 1000 2170540 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516207] [2170541] 1000 2170541 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516209] [2170542] 1000 2170542 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516211] [2170543] 1000 2170543 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516213] [2170544] 1000 2170544 1062 148 49152 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516215] [2170545] 1000 2170545 1062 145 45056 2 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516218] [2170546] 1000 2170546 1062 146 45056 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516220] [2170547] 1000 2170547 1062 148 53248 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516222] [2170548] 1000 2170548 1062 147 40960 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516225] [2170549] 1000 2170549 1062 145 45056 2 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516229] [2170550] 1000 2170550 1062 148 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516233] [2170551] 1000 2170551 1062 148 40960 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516237] [2170552] 1000 2170552 1062 147 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516242] [2170553] 1000 2170553 1062 147 45056 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516246] [2170554] 1000 2170554 1 1 12288 0 0 update_server.s Sep 18 15:41:15 debian-homeserver kernel: [356057.516250] [2170555] 1000 2170555 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516254] [2170556] 1000 2170556 1062 148 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516258] [2170557] 1000 2170557 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516263] [2170558] 1000 2170558 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516267] [2170559] 1000 2170559 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516272] [2170560] 1000 2170560 1 1 12288 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516275] [2170561] 1000 2170561 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516280] [2170562] 1000 2170562 1062 147 45056 1 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516283] [2170563] 1000 2170563 1062 120 40960 0 0 restart_server. Sep 18 15:41:15 debian-homeserver kernel: [356057.516287] [2170564] 1000 2170564 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516291] [2170565] 1000 2170565 1062 148 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516295] [2170566] 1000 2170566 1062 150 45056 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516299] [2170567] 1000 2170567 1062 121 45056 0 0 restart_server. Sep 18 15:41:15 debian-homeserver kernel: [356057.516303] [2170568] 1000 2170568 996 78 45056 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516307] [2170569] 1000 2170569 996 74 40960 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516311] [2170570] 1000 2170570 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516316] [2170571] 1000 2170571 996 77 49152 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516320] [2170572] 1000 2170572 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516324] [2170573] 1000 2170573 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516328] [2170574] 1000 2170574 996 73 40960 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516331] [2170575] 1000 2170575 1029 111 45056 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516333] [2170576] 1000 2170576 996 73 40960 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516336] [2170577] 1000 2170577 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516338] [2170578] 1000 2170578 1062 150 40960 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516340] [2170579] 1000 2170579 1 1 12288 0 0 launch_ASA.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516342] [2170580] 1000 2170580 1029 112 45056 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516345] [2170581] 1000 2170581 996 77 45056 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516347] [2170582] 1000 2170582 1 1 12288 0 0 restart_server. Sep 18 15:41:15 debian-homeserver kernel: [356057.516349] [2170583] 1000 2170583 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516351] [2170584] 1000 2170584 1 1 12288 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516354] [2170585] 1000 2170585 1 1 12288 0 0 restart_server. Sep 18 15:41:15 debian-homeserver kernel: [356057.516356] [2170586] 1000 2170586 1 1 12288 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516359] [2170587] 1000 2170587 996 78 45056 0 0 monitor_ark_ser Sep 18 15:41:15 debian-homeserver kernel: [356057.516364] [2170588] 1000 2170588 996 76 40960 0 0 init.sh Sep 18 15:41:15 debian-homeserver kernel: [356057.516368] [2170589] 1000 2170589 1 1 12288 0 0 launch_ASA.sh