nixos-anywhere icon indicating copy to clipboard operation
nixos-anywhere copied to clipboard

Out of memory with oracle amd 1G free tier

Open mannp opened this issue 1 year ago • 1 comments

Placeholder for more info.

+ printf %s\n 6.1 6.5.0-1023-oracle
+ kexecSyscallFlags=--kexec-syscall-auto
+ sh -c '/root/kexec/kexec/kexec' --load '/root/kexec/kexec/bzImage'   --kexec-syscall-auto      --initrd='/root/kexec/kexec/initrd' --no-checks   --command-line 'init=/nix/store/w5967zp4vrgi8hhsyzb6xv6pv02182j2-nixos-system-nixos-installer-24.05pre-git/init console=tty0 console=ttyS0,115200 root=fstab loglevel=4'
kexec failed, dumping dmesg
Killed
+ echo kexec failed, dumping dmesg
+ dmesg
+ tail -n 100
[ 1549.283759] PEFILE: Unsigned PE binary
[ 1570.685639] kexec invoked oom-killer: gfp_mask=0x100cc2(GFP_HIGHUSER), order=0, oom_score_adj=0
[ 1570.685651] CPU: 0 PID: 3090 Comm: kexec Not tainted 6.5.0-1023-oracle #23~22.04.1-Ubuntu
[ 1570.685657] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.6.6 08/22/2023
[ 1570.685661] Call Trace:
[ 1570.685665]  <TASK>
[ 1570.685669]  dump_stack_lvl+0x48/0x70
[ 1570.685711]  dump_stack+0x10/0x20
[ 1570.685716]  dump_header+0x50/0x270
[ 1570.685724]  oom_kill_process+0x10c/0x1b0
[ 1570.685729]  out_of_memory+0x109/0x350
[ 1570.685735]  __alloc_pages_may_oom+0x114/0x1e0
[ 1570.685743]  __alloc_pages_slowpath.constprop.0+0x4b1/0x9a0
[ 1570.685751]  __alloc_pages+0x2fc/0x330
[ 1570.685758]  alloc_pages+0x95/0x1a0
[ 1570.685764]  kimage_alloc_pages+0x35/0xf0
[ 1570.685770]  kimage_alloc_page+0xc0/0x310
[ 1570.685777]  kimage_load_normal_segment+0xe9/0x270
[ 1570.685783]  kimage_load_segment+0x17/0x30
[ 1570.685788]  __do_sys_kexec_file_load+0x1fc/0x2b0
[ 1570.685795]  __x64_sys_kexec_file_load+0x20/0x40
[ 1570.685800]  x64_sys_call+0x2142/0x2570
[ 1570.685805]  do_syscall_64+0x56/0x90
[ 1570.685811]  ? srso_return_thunk+0x5/0x10
[ 1570.685817]  ? exc_page_fault+0x95/0x190
[ 1570.685823]  entry_SYSCALL_64_after_hwframe+0x73/0xdd
[ 1570.685829] RIP: 0033:0x426efd
[ 1570.685854] Code: Unable to access opcode bytes at 0x426ed3.
[ 1570.685856] RSP: 002b:00007fff125d9540 EFLAGS: 00000212 ORIG_RAX: 0000000000000140
[ 1570.685863] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000426efd
[ 1570.685866] RDX: 0000000000000098 RSI: 0000000000000004 RDI: 0000000000000003
[ 1570.685869] RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
[ 1570.685872] R10: 00007a2f7dba6040 R11: 0000000000000212 R12: 0000000000000003
[ 1570.685875] R13: 00007fff125d9838 R14: 00007fff125d95c0 R15: 0000000000000004
[ 1570.685883]  </TASK>
[ 1570.685886] Mem-Info:
[ 1570.685889] active_anon:11696 inactive_anon:12790 isolated_anon:0
                active_file:14 inactive_file:5 isolated_file:0
                unevictable:3343 dirty:3 writeback:0
                slab_reclaimable:5757 slab_unreclaimable:9108
                mapped:2884 shmem:3015 pagetables:653
                sec_pagetables:0 bounce:0
                kernel_misc_reclaimable:0
                free:12558 free_pcp:307 free_cma:0
[ 1570.685898] Node 0 active_anon:46784kB inactive_anon:51160kB active_file:56kB inactive_file:20kB unevictable:13372kB isolated(anon):0kB isolated(file):0kB mapped:11536kB dirty:12kB writeback:0kB shmem:12060kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB kernel_stack:2528kB pagetables:2612kB sec_pagetables:0kB all_unreclaimable? no
[ 1570.685907] Node 0 DMA free:4116kB boost:0kB min:712kB low:888kB high:1064kB reserved_highatomic:0KB active_anon:0kB inactive_anon:96kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15028kB managed:14392kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:4kB free_cma:0kB
[ 1570.685916] lowmem_reserve[]: 0 873 873 873 873
[ 1570.685929] Node 0 DMA32 free:46116kB boost:0kB min:44340kB low:55424kB high:66508kB reserved_highatomic:2048KB active_anon:31372kB inactive_anon:66476kB active_file:72kB inactive_file:4kB unevictable:13372kB writepending:12kB present:1027952kB managed:955844kB mlocked:13372kB bounce:0kB free_pcp:1216kB local_pcp:460kB free_cma:0kB
[ 1570.685939] lowmem_reserve[]: 0 0 0 0 0
[ 1570.686004] Node 0 DMA: 5*4kB (UM) 6*8kB (UM) 1*16kB (M) 4*32kB (UM) 1*64kB (U) 2*128kB (UM) 2*256kB (UM) 2*512kB (UM) 2*1024kB (M) 0*2048kB 0*4096kB = 4116kB
[ 1570.686047] Node 0 DMA32: 1573*4kB (UMEH) 1038*8kB (UMEH) 640*16kB (UMEH) 321*32kB (UMEH) 126*64kB (UMEH) 19*128kB (UMH) 2*256kB (H) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 46116kB
[ 1570.686086] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[ 1570.686090] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 1570.686093] 5910 total pagecache pages
[ 1570.686096] 0 pages in swap cache
[ 1570.686098] Free swap  = 0kB
[ 1570.686100] Total swap = 0kB
[ 1570.686103] 260745 pages RAM
[ 1570.686105] 0 pages HighMem/MovableOnly
[ 1570.686107] 18186 pages reserved
[ 1570.686109] 0 pages hwpoisoned
[ 1570.686111] Tasks state (memory values in pages):
[ 1570.686113] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[ 1570.686119] [    287]     0   287    16072     1312    94208        0          -250 systemd-journal
[ 1570.686126] [    315]     0   315     6627     1466    81920        0         -1000 systemd-udevd
[ 1570.686133] [    433]   109   433     2026      864    57344        0             0 rpcbind
[ 1570.686138] [    434]   104   434    22341     1216    73728        0             0 systemd-timesyn
[ 1570.686144] [    469]   101   469     4032     1248    69632        0             0 systemd-network
[ 1570.686149] [    471]   102   471     6385     2038    98304        0             0 systemd-resolve
[ 1570.686155] [    680]   103   680     2195      992    57344        0          -900 dbus-daemon
[ 1570.686160] [    681]     0   681     3231      592    65536        0             0 iscsid
[ 1570.686165] [    682]     0   682     3357     3284    65536        0           -17 iscsid
[ 1570.686171] [    687]     0   687    58626      928    86016        0             0 polkitd
[ 1570.686171] [    687]     0   687    58626      928    86016        0             0 polkitd
[ 1570.686176] [    691]     0   691   366867     3298   262144        0          -900 snapd
[ 1570.686181] [    692]     0   692     3875     1248    69632        0             0 systemd-logind
[ 1570.686186] [    695]     0   695    98124     1600   139264        0             0 udisksd
[ 1570.686191] [    752]     0   752    79469     1248   118784        0             0 ModemManager
[ 1570.686196] [    755]     0   755      799      480    49152        0             0 agetty
[ 1570.686201] [    767]     0   767      810      512    49152        0             0 agetty
[ 1570.686206] [    802]     0   802     3860     1888    69632        0         -1000 sshd
[ 1570.686211] [    967] 584788   967   309358     1293   122880        0             0 agent
[ 1570.686216] [    974] 584788   974   309264     1047   122880        0             0 updater
[ 1570.686221] [   1013]     0  1013     4231     2048    73728        0             0 sshd
[ 1570.686226] [   1023] 584788  1023   309668     1839   131072        0             0 gomon
[ 1570.686231] [   1035] 584788  1035   310897     1586   139264        0             0 oci-wlp
[ 1570.686236] [   1037]  1001  1037     4267     1376    73728        0             0 systemd
[ 1570.686241] [   1039]  1001  1039    26167     1373   102400        0             0 (sd-pam)
[ 1570.686247] [   1114]  1001  1114     4371     1723    73728        0             0 sshd
[ 1570.686252] [   2446]     0  2446     4302     2141    77824        0             0 sshd
[ 1570.686257] [   2449]     0  2449     4268     1408    73728        0             0 systemd
[ 1570.686262] [   2450]     0  2450    26199     1374   102400        0             0 (sd-pam)
[ 1570.686267] [   3016]     0  3016     4232     2048    73728        0             0 sshd
[ 1570.686272] [   3059]  1001  3059     4305     1692    73728        0             0 sshd
[ 1570.686277] [   3060]  1001  3060     1229      544    49152        0             0 bash
[ 1570.686282] [   3064]  1001  3064     2234      800    69632        0             0 sudo
[ 1570.686286] [   3065]     0  3065      723      448    45056        0             0 run
[ 1570.686291] [   3089]     0  3089      723      416    45056        0             0 sh
[ 1570.686296] [   3090]     0  3090     3082     2720    65536        0             0 kexec
[ 1570.686301] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1001.slice/session-19.scope,task=kexec,pid=3090,uid=0
[ 1570.686477] Out of memory: Killed process 3090 (kexec) total-vm:12328kB, anon-rss:10880kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:64kB oom_score_adj:0
+ exit 1
+ cleanup
+ rm -rf /root/kexec/kexec/tmp.CzCiBSkk3h
+ rm -rf /tmp/tmp.fjjZC83oJ1

mannp avatar Jul 10 '24 15:07 mannp

Try stopping useless preinstalled services first to reclaim some memory; I've been successful in kexec'ing this way.

t184256 avatar Aug 04 '24 12:08 t184256

If you create the instance with an Oracle Linux 9 image, it will allow kexec to complete without going oom.

jsimonetti avatar Aug 19 '24 11:08 jsimonetti