lxcfs LXCFS crashes, need help diagnosing

trafficstars

Ubuntu Noble LXD v5.21.1 LTS + whatever version of lxcfs is packaged with this

We have a system with about 86 containers and no significant activity, a bit of load but nothing major. Out of our fleet of 60+ hosts which are pretty much identical to this system with some thousands of containers; on this host lxcfs crashes every couple of days. We have checked all logs we can think of, but all we can find is the crash dump itself in the syslog, as seen below.

Any help diagnosing these crashes is much appreciated, we suspect it's tied to some "unique" customer workload as it's only happening here. It's really rather painful having to reboot the entire host and bring down all instances with it, whenever this happens.

Crash data

2024-06-05T14:07:40.930342+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: *** signal 11
2024-06-05T14:07:40.930748+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: Register dump:
2024-06-05T14:07:40.930813+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  RAX: 00000007d89ac102   RBX: ffffffffffffff80   RCX: 00007d8a0e27d769
2024-06-05T14:07:40.930860+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  RDX: 00007d89ed5ffc90   RSI: 00000007d89ac0f2   RDI: 00000007d89ac102
2024-06-05T14:07:40.930920+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  RBP: 00007d89ed5ffb20   R8 : 00007d8a0e262198   R9 : 000000000000000f
2024-06-05T14:07:40.930972+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  R10: 00007d8a0e262198   R11: 0000000000000000   R12: 0000000000000003
2024-06-05T14:07:40.931016+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  R13: 00007d8978107df0   R14: 00006437266e5708   R15: 0000000000000000
2024-06-05T14:07:40.931059+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  RSP: 00007d89ed5ffad0
2024-06-05T14:07:40.931119+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  RIP: 00007d8a0e3493fe   EFLAGS: 00010202
2024-06-05T14:07:40.931163+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  CS: 0033   FS: 0000   GS: 0000
2024-06-05T14:07:40.931230+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  Trap: 0000000e   Error: 00000004   OldMask: 00004007   CR2: d89ac0fa
2024-06-05T14:07:40.931282+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-05T14:07:40.931336+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  RIP: 00000000   RDP: 00000000
2024-06-05T14:07:40.931379+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-05T14:07:40.931430+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-05T14:07:40.931479+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-05T14:07:40.931529+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-05T14:07:40.931578+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  mxcsr: 1fa0
2024-06-05T14:07:40.931628+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-05T14:07:40.931700+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-05T14:07:40.931743+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-05T14:07:40.931794+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-05T14:07:40.931958+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-05T14:07:40.932017+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-05T14:07:40.932066+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-05T14:07:40.932115+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-05T14:07:40.932164+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: Backtrace:
2024-06-05T14:07:40.932228+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /lib/x86_64-linux-gnu/libc.so.6(free+0x1e)[0x7d8a0e3493fe]
2024-06-05T14:07:40.932285+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/liblxcfs.so(do_release_file_info+0x42)[0x7d8a0e2880c8]
2024-06-05T14:07:40.932335+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/liblxcfs.so(proc_release+0x20)[0x7d8a0e27d789]
2024-06-05T14:07:40.932385+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: lxcfs(+0x2f2c)[0x64372561cf2c]
2024-06-05T14:07:40.932434+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: lxcfs(+0x3c7f)[0x64372561dc7f]
2024-06-05T14:07:40.932708+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0xbb6a)[0x7d8a0e4fbb6a]
2024-06-05T14:07:40.932765+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x104fa)[0x7d8a0e5004fa]
2024-06-05T14:07:40.932801+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x11738)[0x7d8a0e501738]
2024-06-05T14:07:40.932892+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x7d8a0e50e13f]
2024-06-05T14:07:40.932957+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x7d8a0e5067a7]
2024-06-05T14:07:40.933000+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7d8a0e338ac3]
2024-06-05T14:07:40.933043+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7d8a0e3ca850]
2024-06-05T14:07:40.933091+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: Memory map:
2024-06-05T14:07:40.933142+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 64372561a000-643725622000 r-xp 00000000 07:01 44                         /snap/lxd/28463/bin/lxcfs
2024-06-05T14:07:40.933216+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 643725622000-643725623000 r--p 00007000 07:01 44                         /snap/lxd/28463/bin/lxcfs
2024-06-05T14:07:40.933275+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 643725623000-643725624000 rw-p 00008000 07:01 44                         /snap/lxd/28463/bin/lxcfs
2024-06-05T14:07:40.933325+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 6437266d8000-643726727000 rw-p 00000000 00:00 0                          [heap]
2024-06-05T14:07:40.933361+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d88f8000000-7d88f8203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.933404+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d88f8203000-7d88fc000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.933447+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8900000000-7d8900204000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.933502+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8900204000-7d8904000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.933551+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8904000000-7d8904203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.933600+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8904203000-7d8908000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.933705+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d890c000000-7d890c203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.933765+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d890c203000-7d8910000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.933817+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8910000000-7d8910213000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.933859+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8910213000-7d8914000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.933908+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8918000000-7d8918206000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.933957+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8918206000-7d891c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934015+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d891c000000-7d891c21d000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934064+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d891c21d000-7d8920000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934121+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8924000000-7d8924177000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934169+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8924177000-7d8928000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934245+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8928000000-7d8928172000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934294+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8928172000-7d892c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934351+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8930000000-7d8930203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934400+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8930203000-7d8934000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934449+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8934000000-7d8934165000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934507+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8934165000-7d8938000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934557+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d893c000000-7d893c17b000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934598+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d893c17b000-7d8940000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934674+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8940000000-7d894016d000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934732+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d894016d000-7d8944000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934789+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8948000000-7d8948207000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.934838+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8948207000-7d894c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.934950+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d894c000000-7d894c203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.935009+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d894c203000-7d8950000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.935723+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8954000000-7d8954203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.935778+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8954203000-7d8958000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.935824+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8958000000-7d895820d000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.935875+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d895820d000-7d895c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.935921+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8960000000-7d8960178000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.935971+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8960178000-7d8964000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936030+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8964000000-7d8964203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936087+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8964203000-7d8968000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936133+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d896c000000-7d896c21a000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936178+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d896c21a000-7d8970000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936247+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8970000000-7d8970204000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936293+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8970204000-7d8974000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936343+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8978000000-7d8978203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936395+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8978203000-7d897c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936446+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d897c000000-7d897c203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936497+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d897c203000-7d8980000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936547+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8984000000-7d898417e000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936597+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d898417e000-7d8988000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936660+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8988000000-7d8988203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936712+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8988203000-7d898c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936763+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8990000000-7d8990203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936818+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8990203000-7d8994000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936875+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8994000000-7d8994203000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.936930+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8994203000-7d8998000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.936975+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d899c000000-7d899c204000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937025+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d899c204000-7d89a0000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.937070+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89a0000000-7d89a0172000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937115+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89a0172000-7d89a4000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.937165+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89a8000000-7d89a8205000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937226+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89a8205000-7d89ac000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.937290+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ac000000-7d89ac17b000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937341+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ac17b000-7d89b0000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.937397+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89b4000000-7d89b416f000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937448+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89b416f000-7d89b8000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.937504+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89b8000000-7d89b8173000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937554+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89b8173000-7d89bc000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.937610+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89c0000000-7d89c0178000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.937983+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89c0178000-7d89c4000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938046+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89c4000000-7d89c417a000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938134+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89c417a000-7d89c8000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938191+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89cc000000-7d89cc204000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938255+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89cc204000-7d89d0000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938306+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89d0000000-7d89d0181000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938363+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89d0181000-7d89d4000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938409+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89d8000000-7d89d8170000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938459+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89d8170000-7d89dc000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938504+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89dc000000-7d89dc173000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938556+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89dc173000-7d89e0000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938602+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89e2c00000-7d89e2c01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938670+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89e2c01000-7d89e3401000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938707+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89e4000000-7d89e4178000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938742+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89e4178000-7d89e8000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938784+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89e8000000-7d89e8173000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938813+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89e8173000-7d89ec000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938848+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ece00000-7d89ece01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938882+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ece01000-7d89ed601000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938916+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ee200000-7d89ee201000 ---p 00000000 00:00 0
2024-06-05T14:07:40.938950+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ee201000-7d89eea01000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.938984+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89eec00000-7d89eec01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939019+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89eec01000-7d89ef401000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939052+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ef600000-7d89ef601000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939086+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89ef601000-7d89efe01000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939120+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89f0000000-7d89f016b000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939148+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89f016b000-7d89f4000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939182+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89f4000000-7d89f4177000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939231+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89f4177000-7d89f8000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939266+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89f8e00000-7d89f8e01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939308+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89f8e01000-7d89f9601000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939346+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89fa200000-7d89fa201000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939387+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89fa201000-7d89faa01000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939421+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89fc000000-7d89fc16c000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939455+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d89fc16c000-7d8a00000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939490+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a00000000-7d8a00175000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939533+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a00175000-7d8a04000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939567+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a04400000-7d8a04401000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939596+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a04401000-7d8a04c01000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939630+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a04e00000-7d8a04e01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939680+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a04e01000-7d8a05601000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939716+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a05800000-7d8a05801000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939762+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a05801000-7d8a06001000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939824+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a06c00000-7d8a06c01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939867+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a06c01000-7d8a07401000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939901+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a07600000-7d8a07601000 ---p 00000000 00:00 0
2024-06-05T14:07:40.939935+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a07601000-7d8a07e01000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.939975+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a08000000-7d8a08067000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940029+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a08067000-7d8a0c000000 ---p 00000000 00:00 0
2024-06-05T14:07:40.940074+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0c600000-7d8a0c601000 ---p 00000000 00:00 0
2024-06-05T14:07:40.940115+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0c601000-7d8a0ce01000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940149+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0d000000-7d8a0d001000 ---p 00000000 00:00 0
2024-06-05T14:07:40.940185+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0d001000-7d8a0d801000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940236+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0d9e3000-7d8a0da00000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940265+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0da00000-7d8a0da01000 ---p 00000000 00:00 0
2024-06-05T14:07:40.940306+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0da01000-7d8a0e201000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940340+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e201000-7d8a0e261000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940376+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e261000-7d8a0e294000 r-xp 00000000 07:01 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-05T14:07:40.940414+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e294000-7d8a0e295000 r--p 00032000 07:01 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-05T14:07:40.940445+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e295000-7d8a0e296000 rw-p 00033000 07:01 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-05T14:07:40.940486+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e296000-7d8a0e2a4000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940530+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e2a4000-7d8a0e2cc000 r--p 00000000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-05T14:07:40.940566+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e2cc000-7d8a0e461000 r-xp 00028000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-05T14:07:40.940601+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e461000-7d8a0e4b9000 r--p 001bd000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-05T14:07:40.940635+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4b9000-7d8a0e4ba000 ---p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-05T14:07:40.940690+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4ba000-7d8a0e4be000 r--p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-05T14:07:40.940726+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4be000-7d8a0e4c0000 rw-p 00219000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-05T14:07:40.940759+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4c0000-7d8a0e4cd000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.940800+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4cd000-7d8a0e4d0000 r--p 00000000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-05T14:07:40.940831+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4d0000-7d8a0e4e7000 r-xp 00003000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-05T14:07:40.940891+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4e7000-7d8a0e4eb000 r--p 0001a000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-05T14:07:40.940956+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4eb000-7d8a0e4ec000 r--p 0001d000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-05T14:07:40.940985+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4ec000-7d8a0e4ed000 rw-p 0001e000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-05T14:07:40.941020+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4ed000-7d8a0e4f0000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.941055+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4f0000-7d8a0e4f7000 r--p 00000000 07:01 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-05T14:07:40.941090+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e4f7000-7d8a0e512000 r-xp 00007000 07:01 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-05T14:07:40.941125+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e512000-7d8a0e51c000 r--p 00022000 07:01 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-05T14:07:40.941349+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e51c000-7d8a0e52e000 r--p 0002b000 07:01 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-05T14:07:40.941386+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e52e000-7d8a0e52f000 rw-p 0003d000 07:01 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-05T14:07:40.941424+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e52f000-7d8a0e530000 r--p 00000000 07:01 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-05T14:07:40.941469+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e530000-7d8a0e533000 r-xp 00001000 07:01 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-05T14:07:40.941506+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e533000-7d8a0e534000 r--p 00004000 07:01 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-05T14:07:40.941546+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e534000-7d8a0e535000 r--p 00005000 07:01 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-05T14:07:40.941581+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e535000-7d8a0e536000 rw-p 00006000 07:01 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-05T14:07:40.941626+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e536000-7d8a0e538000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.941687+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e538000-7d8a0e53a000 r--p 00000000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-05T14:07:40.941724+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e53a000-7d8a0e564000 r-xp 00002000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-05T14:07:40.941780+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e564000-7d8a0e56f000 r--p 0002c000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-05T14:07:40.941817+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e56f000-7d8a0e570000 rw-p 00000000 00:00 0
2024-06-05T14:07:40.941852+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e570000-7d8a0e572000 r--p 00037000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-05T14:07:40.941882+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7d8a0e572000-7d8a0e574000 rw-p 00039000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-05T14:07:40.941975+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7ffd64ec8000-7ffd64ee9000 rw-p 00000000 00:00 0                          [stack]
2024-06-05T14:07:40.942019+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7ffd64f4d000-7ffd64f51000 r--p 00000000 00:00 0                          [vvar]
2024-06-05T14:07:40.942051+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: 7ffd64f51000-7ffd64f53000 r-xp 00000000 00:00 0                          [vdso]
2024-06-05T14:07:40.942081+02:00 lxdvm2-x14-u17-r1 lxd.daemon[5415]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

Jun 05 '24 13:06 webdock-io

@mihalicyn Since you self-assigned I'm hoping this is something which can be looked at and resolved, as we are now seeing this happen on 2 additional systems. What the systems have in common:

They do show some load, but nothing crazy compared to what we've been running in the past without issue
It should be noted this is inside a VM
LXCFS is running with --enable-loadavg

Here is a dump from another machine where this happened, not sure if you can tell if this is an identical crash or different. If it's different, let me know and I'll start collecting crash dumps from the various instances so you can get a complete picture:

Crash data

2024-06-06T23:02:00.051688+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: *** signal 11
2024-06-06T23:02:00.051842+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: Register dump:
2024-06-06T23:02:00.051886+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  RAX: 0000000000008000   RBX: 0000585f1f1ad8c0   RCX: 0000000000000000
2024-06-06T23:02:00.052033+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  RDX: 0000000000004000   RSI: 00007c3774104d30   RDI: 00007c377410e430
2024-06-06T23:02:00.052079+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  RBP: 00007c38241ffae0   R8 : 00007c38241ffc90   R9 : 00007c3824d052fb
2024-06-06T23:02:00.052298+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  R10: 00007c3824ce42a0   R11: 0000000000000000   R12: 00007c377410e430
2024-06-06T23:02:00.052588+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  R13: 00007c38241ffc90   R14: 00007c38241ffbf8   R15: 0000000000000000
2024-06-06T23:02:00.052649+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  RSP: 00007c38241ffaa0
2024-06-06T23:02:00.052669+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  RIP: 00007c3824d0532b   EFLAGS: 00010206
2024-06-06T23:02:00.052696+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  CS: 0033   FS: 0000   GS: 0000
2024-06-06T23:02:00.052756+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  Trap: 0000000e   Error: 00000004   OldMask: 00004007   CR2: 00008018
2024-06-06T23:02:00.052779+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-06T23:02:00.052802+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  RIP: 00000000   RDP: 00000000
2024-06-06T23:02:00.052828+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-06T23:02:00.052854+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-06T23:02:00.052880+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-06T23:02:00.052906+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-06T23:02:00.052948+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  mxcsr: 1fa0
2024-06-06T23:02:00.052972+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-06T23:02:00.052996+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-06T23:02:00.053028+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-06T23:02:00.053062+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-06T23:02:00.053092+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-06T23:02:00.053115+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-06T23:02:00.053136+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-06T23:02:00.053158+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-06T23:02:00.053631+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: Backtrace:
2024-06-06T23:02:00.053715+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /snap/lxd/current/lib/liblxcfs.so(proc_read+0x30)[0x7c3824d0532b]
2024-06-06T23:02:00.053743+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: lxcfs(+0x22c3)[0x585f1e32f2c3]
2024-06-06T23:02:00.053766+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: lxcfs(+0x3a56)[0x585f1e330a56]
2024-06-06T23:02:00.053792+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(fuse_fs_read_buf+0x1be)[0x7c3824f7ff2e]
2024-06-06T23:02:00.053823+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x10d66)[0x7c3824f82d66]
2024-06-06T23:02:00.053854+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x114fa)[0x7c3824f834fa]
2024-06-06T23:02:00.053877+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x7c3824f9013f]
2024-06-06T23:02:00.053903+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x7c3824f887a7]
2024-06-06T23:02:00.053936+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7c3824dbaac3]
2024-06-06T23:02:00.053962+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7c3824e4c850]
2024-06-06T23:02:00.053983+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: Memory map:
2024-06-06T23:02:00.054010+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 585f1e32d000-585f1e335000 r-xp 00000000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-06T23:02:00.054052+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 585f1e335000-585f1e336000 r--p 00007000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-06T23:02:00.054078+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 585f1e336000-585f1e337000 rw-p 00008000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-06T23:02:00.054104+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 585f1f1a0000-585f1f1ef000 rw-p 00000000 00:00 0                          [heap]
2024-06-06T23:02:00.054130+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36e4000000-7c36e4207000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054157+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36e4207000-7c36e8000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054182+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36e8000000-7c36e8203000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054208+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36e8203000-7c36ec000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054237+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36f0000000-7c36f0203000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054262+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36f0203000-7c36f4000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054288+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36f4000000-7c36f4204000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054314+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36f4204000-7c36f8000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054341+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36fc000000-7c36fc208000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054381+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c36fc208000-7c3700000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054430+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3700000000-7c370020d000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054457+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c370020d000-7c3704000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054483+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3708000000-7c3708203000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054508+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3708203000-7c370c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054534+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c370c000000-7c370c220000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054559+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c370c220000-7c3710000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054585+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3714000000-7c3714210000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054610+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3714210000-7c3718000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054635+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3718000000-7c3718208000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054660+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3718208000-7c371c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.054703+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3720000000-7c372021e000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.054733+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c372021e000-7c3724000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.055163+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3724000000-7c3724217000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.055210+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3724217000-7c3728000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.055321+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c372c000000-7c372c20c000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.055391+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c372c20c000-7c3730000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.056914+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3730000000-7c3730221000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057246+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3730221000-7c3734000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057276+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3738000000-7c373816a000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057299+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c373816a000-7c373c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057322+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c373c000000-7c373c20d000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057344+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c373c20d000-7c3740000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057370+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3744000000-7c3744211000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057392+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3744211000-7c3748000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057447+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3748000000-7c374820f000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057473+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c374820f000-7c374c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057499+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3750000000-7c3750211000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057525+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3750211000-7c3754000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057551+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3754000000-7c3754179000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057572+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3754179000-7c3758000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057598+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c375c000000-7c375c206000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057623+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c375c206000-7c3760000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057653+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3760000000-7c3760209000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057678+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3760209000-7c3764000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057703+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3768000000-7c376820b000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057734+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c376820b000-7c376c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057760+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c376c000000-7c376c178000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057785+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c376c178000-7c3770000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057816+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3774000000-7c3774174000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057842+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3774174000-7c3778000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057867+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3778000000-7c377820f000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057892+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c377820f000-7c377c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057917+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3780000000-7c378016c000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.057956+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c378016c000-7c3784000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.057982+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3784000000-7c3784211000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058004+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3784211000-7c3788000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058061+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c378c000000-7c378c20f000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058092+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c378c20f000-7c3790000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058114+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3790000000-7c3790184000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058136+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3790184000-7c3794000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058161+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3798000000-7c37981a3000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058217+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37981a3000-7c379c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058234+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c379c000000-7c379c212000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058256+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c379c212000-7c37a0000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058278+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37a4000000-7c37a418c000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058300+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37a418c000-7c37a8000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058317+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37a8000000-7c37a8227000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058350+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37a8227000-7c37ac000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058372+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37b0000000-7c37b01a1000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058393+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37b01a1000-7c37b4000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058439+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37b4000000-7c37b418a000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058533+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37b418a000-7c37b8000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058558+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37bc000000-7c37bc198000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058575+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37bc198000-7c37c0000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058597+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37c0000000-7c37c018c000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058615+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37c018c000-7c37c4000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058637+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37c8000000-7c37c8222000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058663+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37c8222000-7c37cc000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058685+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37cc000000-7c37cc20e000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058706+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37cc20e000-7c37d0000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058729+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37d4000000-7c37d4195000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058809+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37d4195000-7c37d8000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058836+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37d8000000-7c37d818a000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058861+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37d818a000-7c37dc000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058887+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37e0000000-7c37e0172000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058909+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37e0172000-7c37e4000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058940+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37e4000000-7c37e4194000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.058967+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37e4194000-7c37e8000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.058990+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37ec000000-7c37ec185000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059011+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37ec185000-7c37f0000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059112+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f0000000-7c37f01a8000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059137+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f01a8000-7c37f4000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059159+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f6200000-7c37f6201000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059184+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f6201000-7c37f6a01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059210+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f6c00000-7c37f6c01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059232+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f6c01000-7c37f7401000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059254+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f7600000-7c37f7601000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059276+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f7601000-7c37f7e01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059297+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f8000000-7c37f8191000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059319+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37f8191000-7c37fc000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059340+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37fc000000-7c37fc18a000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059366+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c37fc18a000-7c3800000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059505+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3800400000-7c3800401000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059537+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3800401000-7c3800c01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059564+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3800e00000-7c3800e01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059585+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3800e01000-7c3801601000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059603+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3801800000-7c3801801000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059624+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3801801000-7c3802001000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059645+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3802200000-7c3802201000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059667+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3802201000-7c3802a01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059688+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3802c00000-7c3802c01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059709+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3802c01000-7c3803401000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059736+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3803600000-7c3803601000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059757+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3803601000-7c3803e01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059775+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3804000000-7c3804190000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059801+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3804190000-7c3808000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059824+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3808000000-7c3808060000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059846+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3808060000-7c380c000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059868+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380c400000-7c380c401000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059890+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380c401000-7c380cc01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059908+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380ce00000-7c380ce01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059927+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380ce01000-7c380d601000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059946+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380d800000-7c380d801000 ---p 00000000 00:00 0
2024-06-06T23:02:00.059968+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380d801000-7c380e001000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.059993+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380e200000-7c380e201000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060023+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380e201000-7c380ea01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.060046+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380ec00000-7c380ec01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060067+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380ec01000-7c380f401000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.060163+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380f600000-7c380f601000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060189+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c380f601000-7c380fe01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.060210+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3810000000-7c381018d000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.060228+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381018d000-7c3814000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060249+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3814000000-7c381418d000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.060271+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381418d000-7c3818000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060293+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3818400000-7c3818401000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060318+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3818401000-7c3818c01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.060340+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3818e00000-7c3818e01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.060363+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3818e01000-7c3819601000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061182+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3819800000-7c3819801000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061218+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3819801000-7c381a001000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061245+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381a200000-7c381a201000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061268+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381a201000-7c381aa01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061290+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381ac00000-7c381ac01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061350+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381ac01000-7c381b401000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061373+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381b600000-7c381b601000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061396+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381b601000-7c381be01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061436+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381c000000-7c381c182000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061466+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c381c182000-7c3820000000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061488+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3820800000-7c3820801000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061510+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3820801000-7c3821001000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061541+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3821200000-7c3821201000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061567+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3821201000-7c3821a01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061586+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3821c00000-7c3821c01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061607+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3821c01000-7c3822401000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061629+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3822600000-7c3822601000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061651+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3822601000-7c3822e01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061672+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3823000000-7c3823001000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061693+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3823001000-7c3823801000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061715+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3823a00000-7c3823a01000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061737+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3823a01000-7c3824201000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061759+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824400000-7c3824401000 ---p 00000000 00:00 0
2024-06-06T23:02:00.061784+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824401000-7c3824c01000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061806+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824c66000-7c3824ce3000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061828+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824ce3000-7c3824d16000 r-xp 00000000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-06T23:02:00.061848+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824d16000-7c3824d17000 r--p 00032000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-06T23:02:00.061870+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824d17000-7c3824d18000 rw-p 00033000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-06T23:02:00.061892+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824d18000-7c3824d26000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.061913+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824d26000-7c3824d4e000 r--p 00000000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T23:02:00.061936+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824d4e000-7c3824ee3000 r-xp 00028000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T23:02:00.061959+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824ee3000-7c3824f3b000 r--p 001bd000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T23:02:00.061978+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f3b000-7c3824f3c000 ---p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T23:02:00.062000+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f3c000-7c3824f40000 r--p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T23:02:00.062032+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f40000-7c3824f42000 rw-p 00219000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T23:02:00.062051+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f42000-7c3824f4f000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.062310+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f4f000-7c3824f52000 r--p 00000000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T23:02:00.062338+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f52000-7c3824f69000 r-xp 00003000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T23:02:00.062362+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f69000-7c3824f6d000 r--p 0001a000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T23:02:00.062384+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f6d000-7c3824f6e000 r--p 0001d000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T23:02:00.062414+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f6e000-7c3824f6f000 rw-p 0001e000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T23:02:00.062446+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f6f000-7c3824f72000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.062464+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f72000-7c3824f79000 r--p 00000000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T23:02:00.062487+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f79000-7c3824f94000 r-xp 00007000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T23:02:00.062509+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f94000-7c3824f9e000 r--p 00022000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T23:02:00.062528+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824f9e000-7c3824fb0000 r--p 0002b000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T23:02:00.062547+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb0000-7c3824fb1000 rw-p 0003d000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T23:02:00.062569+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb1000-7c3824fb2000 r--p 00000000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T23:02:00.062596+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb2000-7c3824fb5000 r-xp 00001000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T23:02:00.062615+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb5000-7c3824fb6000 r--p 00004000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T23:02:00.062642+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb6000-7c3824fb7000 r--p 00005000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T23:02:00.062666+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb7000-7c3824fb8000 rw-p 00006000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T23:02:00.062694+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fb8000-7c3824fba000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.062722+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fba000-7c3824fbc000 r--p 00000000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T23:02:00.062762+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fbc000-7c3824fe6000 r-xp 00002000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T23:02:00.062786+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824fe6000-7c3824ff1000 r--p 0002c000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T23:02:00.062811+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824ff1000-7c3824ff2000 rw-p 00000000 00:00 0
2024-06-06T23:02:00.062835+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824ff2000-7c3824ff4000 r--p 00037000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T23:02:00.062860+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7c3824ff4000-7c3824ff6000 rw-p 00039000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T23:02:00.062880+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7ffd7560f000-7ffd75630000 rw-p 00000000 00:00 0                          [stack]
2024-06-06T23:02:00.062908+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7ffd757d0000-7ffd757d4000 r--p 00000000 00:00 0                          [vvar]
2024-06-06T23:02:00.062949+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: 7ffd757d4000-7ffd757d6000 r-xp 00000000 00:00 0                          [vdso]
2024-06-06T23:02:00.062972+02:00 lxdvm2-x14-u15-r1 lxd.daemon[2908]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

Jun 07 '24 06:06 webdock-io

Hi @webdock-io

Thanks for your report! I'm working on your case right now.

Second one is interesting:

00000000000222fb <proc_read@@Base>:
   222fb:       f3 0f 1e fa             endbr64 
   222ff:       55                      push   %rbp
   22300:       48 89 e5                mov    %rsp,%rbp
   22303:       48 83 ec 40             sub    $0x40,%rsp
   
path:
   22307:       48 89 7d e8             mov    %rdi,-0x18(%rbp)

buf:
   2230b:       48 89 75 e0             mov    %rsi,-0x20(%rbp)
   
size:
   2230f:       48 89 55 d8             mov    %rdx,-0x28(%rbp)
   
offset:
   22313:       48 89 4d d0             mov    %rcx,-0x30(%rbp)
   
fi:
   22317:       4c 89 45 c8             mov    %r8,-0x38(%rbp)

rax = fi
   2231b:       48 8b 45 c8             mov    -0x38(%rbp),%rax

rax: fi
rax = *(rax + 16) = fi->fh
   2231f:       48 8b 40 10             mov    0x10(%rax),%rax

rax: fi->fh
*(rbp-8)=rax ( <=> ( f = fi->fh ) )
   22323:       48 89 45 f8             mov    %rax,-0x8(%rbp)

rbp: 7c38241ffae0
rax = *(rbp - 8)
   22327:       48 8b 45 f8             mov    -0x8(%rbp),%rax


rax: 0000000000008000 (invalid pointer!)
eax = *(rax+0x18)=*(rax+16+8) = f->type
   2232b:       8b 40 18                mov    0x18(%rax),%eax <=== BOOM!

offset 2232b <-> virtual address 7c3824d0532b

It looks like for some reason, fi->fh value is wrong (https://github.com/lxc/lxcfs/blob/62c723047ffc7784b097321dd6a7ce893b43a01a/src/proc_fuse.c#L1645).

I'm continuing investigation.

Jun 07 '24 14:06 mihalicyn

@mihalicyn That's amazing! Much appreciated.

I will collect crash dumps from other hosts where this has happened, I think we have seen this across 4 hosts thus far, and post them here when I have the chance - in case they are somehow different from what you've seen here already.

Cheers :)

Jun 07 '24 14:06 webdock-io

Here are the remainder of crash dumps we've seen so far, so you can check if this is the same or different.

Crash data 1

2024-06-03T14:10:00.009431+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: *** signal 11
2024-06-03T14:10:00.009615+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: Register dump:
2024-06-03T14:10:00.009641+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  RAX: 00000007102b410e   RBX: ffffffffffffff80   RCX: 0000710318568769
2024-06-03T14:10:00.009664+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  RDX: 0000710300bffc90   RSI: 00000007102b40fe   RDI: 00000007102b410e
2024-06-03T14:10:00.009686+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  RBP: 0000710300bffb20   R8 : 000071031854d198   R9 : 000000000000000f
2024-06-03T14:10:00.009832+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  R10: 000071031854d198   R11: 0000000000000000   R12: 0000000000000004
2024-06-03T14:10:00.009922+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  R13: 00007101b410e050   R14: 00005f8313fe5718   R15: 0000000000000000
2024-06-03T14:10:00.009945+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  RSP: 0000710300bffad0
2024-06-03T14:10:00.009968+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  RIP: 00007103186343fe   EFLAGS: 00010202
2024-06-03T14:10:00.009993+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  CS: 0033   FS: 0000   GS: 0000
2024-06-03T14:10:00.010018+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  Trap: 0000000e   Error: 00000004   OldMask: 00004007   CR2: 102b4106
2024-06-03T14:10:00.010090+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-03T14:10:00.010116+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  RIP: 00000000   RDP: 00000000
2024-06-03T14:10:00.010138+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-03T14:10:00.010159+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-03T14:10:00.010181+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-03T14:10:00.010206+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-03T14:10:00.010228+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  mxcsr: 1fa0
2024-06-03T14:10:00.010252+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-03T14:10:00.010277+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-03T14:10:00.010307+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-03T14:10:00.010332+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-03T14:10:00.010361+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-03T14:10:00.010384+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-03T14:10:00.010419+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-03T14:10:00.010455+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-03T14:10:00.010479+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: Backtrace:
2024-06-03T14:10:00.012750+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /lib/x86_64-linux-gnu/libc.so.6(free+0x1e)[0x7103186343fe]
2024-06-03T14:10:00.012828+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/liblxcfs.so(do_release_file_info+0x42)[0x7103185730c8]
2024-06-03T14:10:00.012863+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/liblxcfs.so(proc_release+0x20)[0x710318568789]
2024-06-03T14:10:00.012896+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: lxcfs(+0x2f2c)[0x5f8313dc9f2c]
2024-06-03T14:10:00.012933+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: lxcfs(+0x3c7f)[0x5f8313dcac7f]
2024-06-03T14:10:00.012966+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0xbb6a)[0x7103187e6b6a]
2024-06-03T14:10:00.013042+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x104fa)[0x7103187eb4fa]
2024-06-03T14:10:00.013070+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x11738)[0x7103187ec738]
2024-06-03T14:10:00.013092+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x7103187f913f]
2024-06-03T14:10:00.013115+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x7103187f17a7]
2024-06-03T14:10:00.013138+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x710318623ac3]
2024-06-03T14:10:00.013167+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7103186b5850]
2024-06-03T14:10:00.013214+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: Memory map:
2024-06-03T14:10:00.013242+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 5f8313dc7000-5f8313dcf000 r-xp 00000000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-03T14:10:00.013264+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 5f8313dcf000-5f8313dd0000 r--p 00007000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-03T14:10:00.013289+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 5f8313dd0000-5f8313dd1000 rw-p 00008000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-03T14:10:00.013311+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 5f8313fd8000-5f8314027000 rw-p 00000000 00:00 0
         [heap]
2024-06-03T14:10:00.013341+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101b4000000-7101b4163000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013434+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101b4163000-7101b8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013466+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101b8000000-7101b8156000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013494+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101b8156000-7101bc000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013519+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101c0000000-7101c015b000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013541+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101c015b000-7101c4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013565+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101c4000000-7101c415d000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013590+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101c415d000-7101c8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013615+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101cc000000-7101cc107000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013639+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101cc107000-7101d0000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013664+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101d0000000-7101d0153000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013688+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101d0153000-7101d4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013712+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101d8000000-7101d811e000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013732+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101d811e000-7101dc000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013755+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101dc000000-7101dc14e000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013779+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101dc14e000-7101e0000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013804+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101e4000000-7101e4116000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013832+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101e4116000-7101e8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013857+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101e8000000-7101e814f000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013883+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101e814f000-7101ec000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013907+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101f0000000-7101f011b000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013932+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101f011b000-7101f4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.013956+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101f4000000-7101f415d000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.013981+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101f415d000-7101f8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014006+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101fc000000-7101fc120000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014042+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7101fc120000-710200000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014067+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710200000000-71020014f000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014088+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71020014f000-710204000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014112+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710208000000-710208166000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014135+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710208166000-71020c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014160+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71020c000000-71020c15b000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014184+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71020c15b000-710210000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014213+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710214000000-71021413b000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014256+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71021413b000-710218000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014282+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710218000000-71021815a000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014302+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71021815a000-71021c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014327+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710220000000-710220203000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014350+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710220203000-710224000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014375+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710224000000-710224135000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014399+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710224135000-710228000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014433+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71022c000000-71022c15c000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014460+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71022c15c000-710230000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014489+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710230000000-71023015a000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014514+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71023015a000-710234000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014535+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710238000000-710238158000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014586+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710238158000-71023c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014632+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71023c000000-71023c156000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014661+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71023c156000-710240000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014686+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710244000000-710244203000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014707+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710244203000-710248000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014732+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710248000000-710248205000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014756+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710248205000-71024c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014780+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710250000000-710250151000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014804+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710250151000-710254000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014828+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710254000000-71025415e000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014853+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71025415e000-710258000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014878+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71025c000000-71025c204000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014901+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71025c204000-710260000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014926+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710260000000-710260156000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.014951+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710260156000-710264000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.014976+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710268000000-710268228000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.015005+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710268228000-71026c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.018810+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71026c000000-71026c11d000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.018983+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71026c11d000-710270000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019016+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710274000000-710274150000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019080+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710274150000-710278000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019127+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710278000000-71027815d000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019152+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71027815d000-71027c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019177+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710280000000-71028015e000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019202+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71028015e000-710284000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019227+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710284000000-710284166000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019332+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710284166000-710288000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019365+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71028c000000-71028c162000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019387+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71028c162000-710290000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019454+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710290000000-710290172000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019495+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710290172000-710294000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019516+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710298000000-710298156000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019541+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710298156000-71029c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019576+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71029c000000-71029c15e000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019604+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71029c15e000-7102a0000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019626+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102a4000000-7102a4154000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019651+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102a4154000-7102a8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019675+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102a8000000-7102a8162000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019726+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102a8162000-7102ac000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019751+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102b0000000-7102b0206000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019775+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102b0206000-7102b4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019800+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102b4000000-7102b415d000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019829+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102b415d000-7102b8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019854+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102bc000000-7102bc15d000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019878+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102bc15d000-7102c0000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019903+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102c0000000-7102c015c000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019927+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102c015c000-7102c4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.019948+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102c8000000-7102c8160000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.019978+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102c8160000-7102cc000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020003+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102cc000000-7102cc168000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020043+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102cc168000-7102d0000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020068+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102d4000000-7102d4154000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020124+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102d4154000-7102d8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020149+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102d8000000-7102d8134000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020171+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102d8134000-7102dc000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020196+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102e0000000-7102e0204000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020220+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102e0204000-7102e4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020244+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102e4000000-7102e414f000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020270+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102e414f000-7102e8000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020301+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102ec000000-7102ec160000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020330+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102ec160000-7102f0000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020356+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f0000000-7102f0158000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020395+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f0158000-7102f4000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020443+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f4e00000-7102f4e01000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020532+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f4e01000-7102f5601000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020557+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f6200000-7102f6201000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020581+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f6201000-7102f6a01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020609+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f6c00000-7102f6c01000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020672+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f6c01000-7102f7401000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020815+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f8000000-7102f8154000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020864+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102f8154000-7102fc000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020888+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102fc000000-7102fc062000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020911+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7102fc062000-710300000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020935+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710300400000-710300401000 ---p 00000000 00:00 0
2024-06-03T14:10:00.020959+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710300401000-710300c01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.020983+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710301800000-710301801000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021008+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710301801000-710302001000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021049+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710302200000-710302201000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021069+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710302201000-710302a01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021093+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710302c00000-710302c01000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021116+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710302c01000-710303401000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021140+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710303600000-710303601000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021171+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710303601000-710303e01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021193+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710304000000-71030414f000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021214+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030414f000-710308000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021235+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710308000000-71030812f000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021257+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030812f000-71030c000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021278+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030ce00000-71030ce01000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021299+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030ce01000-71030d601000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021320+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030ec00000-71030ec01000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021341+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030ec01000-71030f401000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021362+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030f600000-71030f601000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021387+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71030f601000-71030fe01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021410+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710310000000-710310162000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021447+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710310162000-710314000000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021464+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710315400000-710315401000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021485+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710315401000-710315c01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021507+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710316800000-710316801000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021528+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710316801000-710317001000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021550+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710317200000-710317201000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021575+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710317201000-710317a01000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021593+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710317c00000-710317c01000 ---p 00000000 00:00 0
2024-06-03T14:10:00.021614+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710317c01000-710318401000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021638+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318448000-71031844a000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021660+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103184d1000-71031854c000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021681+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031854c000-71031857f000 r-xp 00000000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-03T14:10:00.021713+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031857f000-710318580000 r--p 00032000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-03T14:10:00.021736+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318580000-710318581000 rw-p 00033000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-03T14:10:00.021774+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318581000-71031858f000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021796+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031858f000-7103185b7000 r--p 00000000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-03T14:10:00.021818+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103185b7000-71031874c000 r-xp 00028000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-03T14:10:00.021843+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031874c000-7103187a4000 r--p 001bd000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-03T14:10:00.021861+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187a4000-7103187a5000 ---p 00215000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-03T14:10:00.021882+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187a5000-7103187a9000 r--p 00215000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-03T14:10:00.021904+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187a9000-7103187ab000 rw-p 00219000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-03T14:10:00.021926+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187ab000-7103187b8000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.021947+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187b8000-7103187bb000 r--p 00000000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-03T14:10:00.021969+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187bb000-7103187d2000 r-xp 00003000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-03T14:10:00.021991+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187d2000-7103187d6000 r--p 0001a000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-03T14:10:00.024456+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187d6000-7103187d7000 r--p 0001d000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-03T14:10:00.024629+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187d7000-7103187d8000 rw-p 0001e000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-03T14:10:00.024682+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187d8000-7103187db000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.024794+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187db000-7103187e2000 r--p 00000000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-03T14:10:00.024833+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187e2000-7103187fd000 r-xp 00007000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-03T14:10:00.024867+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7103187fd000-710318807000 r--p 00022000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-03T14:10:00.024901+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318807000-710318819000 r--p 0002b000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-03T14:10:00.024964+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318819000-71031881a000 rw-p 0003d000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-03T14:10:00.025067+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031881a000-71031881b000 r--p 00000000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-03T14:10:00.025101+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031881b000-71031881e000 r-xp 00001000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-03T14:10:00.025127+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031881e000-71031881f000 r--p 00004000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-03T14:10:00.025152+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031881f000-710318820000 r--p 00005000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-03T14:10:00.025178+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318820000-710318821000 rw-p 00006000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-03T14:10:00.025203+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318821000-710318823000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.025225+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318823000-710318825000 r--p 00000000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-03T14:10:00.025250+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 710318825000-71031884f000 r-xp 00002000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-03T14:10:00.025280+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031884f000-71031885a000 r--p 0002c000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-03T14:10:00.025301+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031885a000-71031885b000 rw-p 00000000 00:00 0
2024-06-03T14:10:00.025326+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031885b000-71031885d000 r--p 00037000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-03T14:10:00.025344+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 71031885d000-71031885f000 rw-p 00039000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-03T14:10:00.025366+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7ffe62a68000-7ffe62a89000 rw-p 00000000 00:00 0
         [stack]
2024-06-03T14:10:00.025393+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7ffe62bc9000-7ffe62bcd000 r--p 00000000 00:00 0
         [vvar]
2024-06-03T14:10:00.025423+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: 7ffe62bcd000-7ffe62bcf000 r-xp 00000000 00:00 0
         [vdso]
2024-06-03T14:10:00.025448+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2883]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0
         [vsyscall]

Crash data 2

2024-06-06T08:40:00.009928+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: *** signal 11
2024-06-06T08:40:00.010279+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: Register dump:
2024-06-06T08:40:00.010316+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  RAX: 00000007aba80103   RBX: ffffffffffffff80   RCX: 00007abb22b2e769
2024-06-06T08:40:00.010343+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  RDX: 00007abb215ffc90   RSI: 00000007aba800f3   RDI: 00000007aba80103
2024-06-06T08:40:00.010373+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  RBP: 00007abb215ffb20   R8 : 00007abb22b13198   R9 : 000000000000000f
2024-06-06T08:40:00.010422+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  R10: 00007abb22b13198   R11: 0000000000000000   R12: 0000000000000008
2024-06-06T08:40:00.010451+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  R13: 00007ababc106b70   R14: 00005c3e09e8c6f8   R15: 0000000000000000
2024-06-06T08:40:00.010479+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  RSP: 00007abb215ffad0
2024-06-06T08:40:00.010508+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  RIP: 00007abb22bfa3fe   EFLAGS: 00010202
2024-06-06T08:40:00.010545+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  CS: 0033   FS: 0000   GS: 0000
2024-06-06T08:40:00.010578+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  Trap: 0000000e   Error: 00000004   OldMask: 00004007   CR2: aba800fb
2024-06-06T08:40:00.010603+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-06T08:40:00.010630+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  RIP: 00000000   RDP: 00000000
2024-06-06T08:40:00.010658+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-06T08:40:00.010685+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-06T08:40:00.010712+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-06T08:40:00.010738+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-06T08:40:00.010765+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  mxcsr: 1fa0
2024-06-06T08:40:00.010793+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-06T08:40:00.010821+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-06T08:40:00.010849+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-06T08:40:00.010876+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-06T08:40:00.010904+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-06T08:40:00.010936+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-06T08:40:00.010970+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-06T08:40:00.010997+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-06T08:40:00.011024+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: Backtrace:
2024-06-06T08:40:00.011111+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /lib/x86_64-linux-gnu/libc.so.6(free+0x1e)[0x7abb22bfa3fe]
2024-06-06T08:40:00.011149+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/liblxcfs.so(do_release_file_info+0x42)[0x7abb22b390c8]
2024-06-06T08:40:00.011178+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/liblxcfs.so(proc_release+0x20)[0x7abb22b2e789]
2024-06-06T08:40:00.011205+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: lxcfs(+0x2f2c)[0x5c3e0903bf2c]
2024-06-06T08:40:00.011229+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: lxcfs(+0x3c7f)[0x5c3e0903cc7f]
2024-06-06T08:40:00.011256+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0xbb6a)[0x7abb22dacb6a]
2024-06-06T08:40:00.011284+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x104fa)[0x7abb22db14fa]
2024-06-06T08:40:00.011311+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x11738)[0x7abb22db2738]
2024-06-06T08:40:00.011339+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x7abb22dbf13f]
2024-06-06T08:40:00.011370+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x7abb22db77a7]
2024-06-06T08:40:00.012460+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7abb22be9ac3]
2024-06-06T08:40:00.012534+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7abb22c7b850]
2024-06-06T08:40:00.012573+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: Memory map:
2024-06-06T08:40:00.012616+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 5c3e09039000-5c3e09041000 r-xp 00000000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-06T08:40:00.012654+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 5c3e09041000-5c3e09042000 r--p 00007000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-06T08:40:00.012696+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 5c3e09042000-5c3e09043000 rw-p 00008000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-06T08:40:00.012733+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 5c3e09e7f000-5c3e09ece000 rw-p 00000000 00:00 0
         [heap]
2024-06-06T08:40:00.012774+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba60000000-7aba6014b000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.012814+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba6014b000-7aba64000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.012854+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba68000000-7aba68156000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.012895+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba68156000-7aba6c000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.012934+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba6c000000-7aba6c158000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.012975+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba6c158000-7aba70000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013011+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba74000000-7aba74154000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.013050+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba74154000-7aba78000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013116+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba78000000-7aba78203000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.013158+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba78203000-7aba7c000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013199+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba80000000-7aba80161000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.013660+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba80161000-7aba84000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013720+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba84000000-7aba8415b000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.013763+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba8415b000-7aba88000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013800+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba8c000000-7aba8c203000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.013836+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba8c203000-7aba90000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013873+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba90000000-7aba9015f000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.013909+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba9015f000-7aba94000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.013974+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba98000000-7aba9815a000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014011+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba9815a000-7aba9c000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014052+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba9c000000-7aba9c157000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014106+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7aba9c157000-7abaa0000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014143+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaa4000000-7abaa415f000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014185+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaa415f000-7abaa8000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014222+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaa8000000-7abaa8156000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014345+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaa8156000-7abaac000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014382+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abab0000000-7abab015a000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014435+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abab015a000-7abab4000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014472+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abab4000000-7abab4203000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014508+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abab4203000-7abab8000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014543+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7ababc000000-7ababc205000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014580+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7ababc205000-7abac0000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014621+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abac0000000-7abac015b000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014657+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abac015b000-7abac4000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014693+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abac8000000-7abac8230000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014730+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abac8230000-7abacc000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014765+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abacc000000-7abacc15e000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014807+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abacc15e000-7abad0000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014849+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abad4000000-7abad4166000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014890+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abad4166000-7abad8000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014926+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abad8000000-7abad8151000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.014962+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abad8151000-7abadc000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.014998+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abae0000000-7abae005f000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015035+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abae005f000-7abae4000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015083+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abae4000000-7abae4154000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015119+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abae4154000-7abae8000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015159+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abae9800000-7abae9801000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015195+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abae9801000-7abaea001000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015233+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaec000000-7abaec156000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015323+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaec156000-7abaf0000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015360+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaf0000000-7abaf0165000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015409+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaf0165000-7abaf4000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015447+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaf8000000-7abaf8163000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015489+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abaf8163000-7abafc000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015526+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abafc000000-7abafc166000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015562+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abafc166000-7abb00000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015598+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb00e00000-7abb00e01000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015634+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb00e01000-7abb01601000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015671+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb01800000-7abb01801000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015734+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb01801000-7abb02001000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015777+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb02200000-7abb02201000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015819+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb02201000-7abb02a01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015860+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb02c00000-7abb02c01000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015903+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb02c01000-7abb03401000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.015948+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb03600000-7abb03601000 ---p 00000000 00:00 0
2024-06-06T08:40:00.015988+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb03601000-7abb03e01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016027+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb04000000-7abb04163000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016091+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb04163000-7abb08000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016140+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb08000000-7abb08170000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016178+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb08170000-7abb0c000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016218+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb0e200000-7abb0e201000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016315+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb0e201000-7abb0ea01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016356+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb0f600000-7abb0f601000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016413+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb0f601000-7abb0fe01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016461+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb10000000-7abb10159000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016502+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb10159000-7abb14000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016541+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb14000000-7abb14155000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016582+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb14155000-7abb18000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016618+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb18e00000-7abb18e01000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016661+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb18e01000-7abb19601000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016698+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb19800000-7abb19801000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016739+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb19801000-7abb1a001000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016775+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb1a200000-7abb1a201000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016816+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb1a201000-7abb1aa01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016853+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb1b600000-7abb1b601000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016888+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb1b601000-7abb1be01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016921+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb1c000000-7abb1c161000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.016956+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb1c161000-7abb20000000 ---p 00000000 00:00 0
2024-06-06T08:40:00.016992+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb20e00000-7abb20e01000 ---p 00000000 00:00 0
2024-06-06T08:40:00.017029+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb20e01000-7abb21601000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017081+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb21800000-7abb21801000 ---p 00000000 00:00 0
2024-06-06T08:40:00.017120+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb21801000-7abb22001000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017158+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22200000-7abb22201000 ---p 00000000 00:00 0
2024-06-06T08:40:00.017190+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22201000-7abb22a01000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017222+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22a0e000-7abb22a10000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017317+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22a97000-7abb22b12000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017360+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22b12000-7abb22b45000 r-xp 00000000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-06T08:40:00.017407+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22b45000-7abb22b46000 r--p 00032000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-06T08:40:00.017477+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22b46000-7abb22b47000 rw-p 00033000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-06T08:40:00.017518+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22b47000-7abb22b55000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017556+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22b55000-7abb22b7d000 r--p 00000000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T08:40:00.017594+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22b7d000-7abb22d12000 r-xp 00028000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T08:40:00.017631+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d12000-7abb22d6a000 r--p 001bd000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T08:40:00.017669+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d6a000-7abb22d6b000 ---p 00215000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T08:40:00.017706+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d6b000-7abb22d6f000 r--p 00215000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T08:40:00.017743+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d6f000-7abb22d71000 rw-p 00219000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-06T08:40:00.017779+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d71000-7abb22d7e000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.017817+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d7e000-7abb22d81000 r--p 00000000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T08:40:00.017855+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d81000-7abb22d98000 r-xp 00003000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T08:40:00.017912+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d98000-7abb22d9c000 r--p 0001a000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T08:40:00.018025+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d9c000-7abb22d9d000 r--p 0001d000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T08:40:00.018090+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d9d000-7abb22d9e000 rw-p 0001e000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-06T08:40:00.018134+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22d9e000-7abb22da1000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.018176+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22da1000-7abb22da8000 r--p 00000000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T08:40:00.018218+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22da8000-7abb22dc3000 r-xp 00007000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T08:40:00.018319+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22dc3000-7abb22dcd000 r--p 00022000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T08:40:00.018359+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22dcd000-7abb22ddf000 r--p 0002b000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T08:40:00.018433+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22ddf000-7abb22de0000 rw-p 0003d000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-06T08:40:00.018476+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de0000-7abb22de1000 r--p 00000000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T08:40:00.018518+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de1000-7abb22de4000 r-xp 00001000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T08:40:00.018559+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de4000-7abb22de5000 r--p 00004000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T08:40:00.018851+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de5000-7abb22de6000 r--p 00005000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T08:40:00.018912+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de6000-7abb22de7000 rw-p 00006000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-06T08:40:00.018953+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de7000-7abb22de9000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.018997+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22de9000-7abb22deb000 r--p 00000000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T08:40:00.019032+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22deb000-7abb22e15000 r-xp 00002000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T08:40:00.019084+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22e15000-7abb22e20000 r--p 0002c000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T08:40:00.019125+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22e20000-7abb22e21000 rw-p 00000000 00:00 0
2024-06-06T08:40:00.019433+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22e21000-7abb22e23000 r--p 00037000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T08:40:00.019656+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7abb22e23000-7abb22e25000 rw-p 00039000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-06T08:40:00.019692+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7ffc17715000-7ffc17736000 rw-p 00000000 00:00 0
         [stack]
2024-06-06T08:40:00.019724+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7ffc1774a000-7ffc1774e000 r--p 00000000 00:00 0
         [vvar]
2024-06-06T08:40:00.019749+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: 7ffc1774e000-7ffc17750000 r-xp 00000000 00:00 0
         [vdso]
2024-06-06T08:40:00.019774+02:00 lxdvm2-x14-u26-r1 lxd.daemon[2878]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0
         [vsyscall]

Crash data 3

2024-06-07T07:59:00.045421+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: *** signal 11
2024-06-07T07:59:00.045640+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: Register dump:
2024-06-07T07:59:00.045691+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  RAX: 00007320b70e8300   RBX: 0000000000000000   RCX: 0000000000000000
2024-06-07T07:59:00.045726+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  RDX: 0000000000000222   RSI: 0000000000000000   RDI: 00007320a15ff750
2024-06-07T07:59:00.045750+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  RBP: 0000000000000000   R8 : 0000000000000000   R9 : 00007320a15ff840
2024-06-07T07:59:00.045783+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  R10: 0000000000000003   R11: 000000000000003f   R12: 0000000000000222
2024-06-07T07:59:00.045807+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  R13: 00007320a15ff750   R14: 00007320b6eba988   R15: 00007320a15ff8d0
2024-06-07T07:59:00.045830+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  RSP: 00007320a15ff740
2024-06-07T07:59:00.045853+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  RIP: 00007320b6f59481   EFLAGS: 00010206
2024-06-07T07:59:00.045877+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  CS: 0033   FS: 0000   GS: 0000
2024-06-07T07:59:00.045901+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  Trap: 0000000e   Error: 00000006   OldMask: 00004007   CR2: 00000000
2024-06-07T07:59:00.045930+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-07T07:59:00.045958+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  RIP: 00000000   RDP: 00000000
2024-06-07T07:59:00.045981+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-07T07:59:00.046010+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-07T07:59:00.046034+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-07T07:59:00.046062+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-07T07:59:00.046090+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  mxcsr: 1fa0
2024-06-07T07:59:00.046113+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-07T07:59:00.046142+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-07T07:59:00.046166+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-07T07:59:00.046191+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-07T07:59:00.046219+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-07T07:59:00.046242+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-07T07:59:00.046291+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-07T07:59:00.046319+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-07T07:59:00.046343+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: Backtrace:
2024-06-07T07:59:00.046366+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /lib/x86_64-linux-gnu/libc.so.6(+0x88481)[0x7320b6f59481]
2024-06-07T07:59:00.046390+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /lib/x86_64-linux-gnu/libc.so.6(snprintf+0x96)[0x7320b6f31856]
2024-06-07T07:59:00.046418+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/liblxcfs.so(proc_loadavg_read+0x49b)[0x7320b6eb0f4a]
2024-06-07T07:59:00.046442+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/liblxcfs.so(proc_read+0x26d)[0x7320b6eb0568]
2024-06-07T07:59:00.046469+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: lxcfs(+0x22c3)[0x6346fe2a22c3]
2024-06-07T07:59:00.046492+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: lxcfs(+0x3a56)[0x6346fe2a3a56]
2024-06-07T07:59:00.046517+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(fuse_fs_read_buf+0x1be)[0x7320b712af2e]
2024-06-07T07:59:00.046541+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x10d66)[0x7320b712dd66]
2024-06-07T07:59:00.046607+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x114fa)[0x7320b712e4fa]
2024-06-07T07:59:00.046632+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x7320b713b13f]
2024-06-07T07:59:00.046659+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x7320b71337a7]
2024-06-07T07:59:00.046682+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7320b6f65ac3]
2024-06-07T07:59:00.046705+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7320b6ff7850]
2024-06-07T07:59:00.046729+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: Memory map:
2024-06-07T07:59:00.046758+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 6346fe2a0000-6346fe2a8000 r-xp 00000000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-07T07:59:00.046782+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 6346fe2a8000-6346fe2a9000 r--p 00007000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-07T07:59:00.046805+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 6346fe2a9000-6346fe2aa000 rw-p 00008000 07:01 44
         /snap/lxd/28463/bin/lxcfs
2024-06-07T07:59:00.046829+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 6346ff86f000-6346ff8be000 rw-p 00000000 00:00 0
         [heap]
2024-06-07T07:59:00.046852+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fb4000000-731fb414c000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.046881+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fb414c000-731fb8000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.046910+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fb8000000-731fb8111000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.046933+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fb8111000-731fbc000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.046957+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fc0000000-731fc0113000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.046980+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fc0113000-731fc4000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.047009+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fc4000000-731fc4143000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.047037+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fc4143000-731fc8000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.047060+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fcc000000-731fcc158000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.047083+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fcc158000-731fd0000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.047110+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fd0000000-731fd0156000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.047129+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fd0156000-731fd4000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.047157+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fd8000000-731fd8131000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.047183+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fd8131000-731fdc000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.047206+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fdc000000-731fdc12c000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.047229+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fdc12c000-731fe0000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.047253+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fe4000000-731fe413c000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.056702+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fe413c000-731fe8000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.056795+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fe8000000-731fe812b000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.056860+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731fe812b000-731fec000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.056923+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731ff0000000-731ff0159000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.064509+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731ff0159000-731ff4000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.064629+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731ff4000000-731ff4115000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.064699+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731ff4115000-731ff8000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.064769+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731ffc000000-731ffc116000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.064832+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 731ffc116000-732000000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.064926+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732000000000-73200014d000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.064996+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73200014d000-732004000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.065059+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732008000000-732008154000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.065121+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732008154000-73200c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.067642+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73200c000000-73200c203000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.067720+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73200c203000-732010000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.067790+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732014000000-732014114000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.067835+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732014114000-732018000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.067867+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732018000000-73201811f000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.067891+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73201811f000-73201c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.067919+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732020000000-732020150000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.067942+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732020150000-732024000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.067968+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732024000000-73202411e000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.067987+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73202411e000-732028000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068037+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73202c000000-73202c147000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068132+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73202c147000-732030000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068161+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732030000000-73203011a000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068236+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73203011a000-732034000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068359+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732038000000-73203811c000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068419+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73203811c000-73203c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068480+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73203c000000-73203c124000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068556+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73203c124000-732040000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068619+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732044000000-732044204000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068687+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732044204000-732048000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068748+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732048000000-732048203000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068804+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732048203000-73204c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068873+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732050000000-73205011f000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.068929+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73205011f000-732054000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.068995+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732054000000-732054142000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.069055+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732054142000-732058000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.069116+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73205c000000-73205c21e000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.069177+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73205c21e000-732060000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.069240+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732060000000-73206021a000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.071897+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73206021a000-732064000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.071935+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732068000000-732068152000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.071956+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732068152000-73206c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.071987+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73206c000000-73206c147000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072011+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73206c147000-732070000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072038+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732074000000-732074112000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072071+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732074112000-732078000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072098+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732078000000-732078205000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072161+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732078205000-73207c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072194+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732080000000-732080116000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072221+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732080116000-732084000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072248+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732084000000-73208405e000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072315+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73208405e000-732088000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072348+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73208c000000-73208c222000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072380+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73208c222000-732090000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072403+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732090000000-732090172000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072423+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732090172000-732094000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072454+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732096c00000-732096c01000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072487+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732096c01000-732097401000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072514+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732097600000-732097601000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072540+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732097601000-732097e01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072573+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 732098000000-73209822c000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072600+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73209822c000-73209c000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072631+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73209c000000-73209c11d000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072658+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 73209c11d000-7320a0000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072685+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a0400000-7320a0401000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072711+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a0401000-7320a0c01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072738+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a0e00000-7320a0e01000 ---p 00000000 00:00 0
2024-06-07T07:59:00.072765+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a0e01000-7320a1601000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.072796+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a1800000-7320a1801000 ---p 00000000 00:00 0
2024-06-07T07:59:00.075589+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a1801000-7320a2001000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.075653+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a2200000-7320a2201000 ---p 00000000 00:00 0
2024-06-07T07:59:00.075692+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a2201000-7320a2a01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.075742+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a2c00000-7320a2c01000 ---p 00000000 00:00 0
2024-06-07T07:59:00.075797+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a2c01000-7320a3401000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.075848+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a3600000-7320a3601000 ---p 00000000 00:00 0
2024-06-07T07:59:00.075894+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a3601000-7320a3e01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.075937+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a4000000-7320a415a000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.075980+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a415a000-7320a8000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076028+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a8000000-7320a8140000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076070+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320a8140000-7320ac000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076118+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ac400000-7320ac401000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076161+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ac401000-7320acc01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076204+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ace00000-7320ace01000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076242+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ace01000-7320ad601000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076330+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ad800000-7320ad801000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076374+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ad801000-7320ae001000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076424+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ae200000-7320ae201000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076465+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320ae201000-7320aea01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076551+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320aec00000-7320aec01000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076615+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320aec01000-7320af401000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076658+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320af600000-7320af601000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076704+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320af601000-7320afe01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076741+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b0000000-7320b0160000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076778+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b0160000-7320b4000000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076817+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b4800000-7320b4801000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076857+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b4801000-7320b5001000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076902+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b5200000-7320b5201000 ---p 00000000 00:00 0
2024-06-07T07:59:00.076944+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b5201000-7320b5a01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.076985+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b5c00000-7320b5c01000 ---p 00000000 00:00 0
2024-06-07T07:59:00.077035+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b5c01000-7320b6401000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.077073+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6600000-7320b6601000 ---p 00000000 00:00 0
2024-06-07T07:59:00.078917+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6601000-7320b6e01000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.078952+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6e76000-7320b6e8e000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.079008+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6e8e000-7320b6ec1000 r-xp 00000000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-07T07:59:00.079035+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6ec1000-7320b6ec2000 r--p 00032000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-07T07:59:00.079059+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6ec2000-7320b6ec3000 rw-p 00033000 07:01 155
         /snap/lxd/28463/lib/liblxcfs.so
2024-06-07T07:59:00.079083+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6ec3000-7320b6ed1000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.079107+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6ed1000-7320b6ef9000 r--p 00000000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-07T07:59:00.079132+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b6ef9000-7320b708e000 r-xp 00028000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-07T07:59:00.079155+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b708e000-7320b70e6000 r--p 001bd000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-07T07:59:00.079184+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b70e6000-7320b70e7000 ---p 00215000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-07T07:59:00.079212+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b70e7000-7320b70eb000 r--p 00215000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-07T07:59:00.079241+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b70eb000-7320b70ed000 rw-p 00219000 07:00 7471
         /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-07T07:59:00.083428+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b70ed000-7320b70fa000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.083492+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b70fa000-7320b70fd000 r--p 00000000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-07T07:59:00.083532+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b70fd000-7320b7114000 r-xp 00003000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-07T07:59:00.083587+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7114000-7320b7118000 r--p 0001a000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-07T07:59:00.083615+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7118000-7320b7119000 r--p 0001d000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-07T07:59:00.083646+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7119000-7320b711a000 rw-p 0001e000 07:00 7521
         /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-07T07:59:00.083671+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b711a000-7320b711d000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.083698+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b711d000-7320b7124000 r--p 00000000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-07T07:59:00.083725+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7124000-7320b713f000 r-xp 00007000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-07T07:59:00.083756+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b713f000-7320b7149000 r--p 00022000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-07T07:59:00.083781+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7149000-7320b715b000 r--p 0002b000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-07T07:59:00.083807+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b715b000-7320b715c000 rw-p 0003d000 07:01 565
         /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-07T07:59:00.083837+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b715c000-7320b715d000 r--p 00000000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-07T07:59:00.083867+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b715d000-7320b7160000 r-xp 00001000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-07T07:59:00.083899+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7160000-7320b7161000 r--p 00004000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-07T07:59:00.083927+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7161000-7320b7162000 r--p 00005000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-07T07:59:00.083952+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7162000-7320b7163000 rw-p 00006000 07:01 538
         /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-07T07:59:00.083978+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7163000-7320b7165000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.084000+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7165000-7320b7167000 r--p 00000000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-07T07:59:00.084031+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7167000-7320b7191000 r-xp 00002000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-07T07:59:00.084072+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b7191000-7320b719c000 r--p 0002c000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-07T07:59:00.084097+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b719c000-7320b719d000 rw-p 00000000 00:00 0
2024-06-07T07:59:00.084122+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b719d000-7320b719f000 r--p 00037000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-07T07:59:00.084152+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7320b719f000-7320b71a1000 rw-p 00039000 07:00 7442
         /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-07T07:59:00.084176+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7ffd2a921000-7ffd2a942000 rw-p 00000000 00:00 0
         [stack]
2024-06-07T07:59:00.084202+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7ffd2a9d3000-7ffd2a9d7000 r--p 00000000 00:00 0
         [vvar]
2024-06-07T07:59:00.084227+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: 7ffd2a9d7000-7ffd2a9d9000 r-xp 00000000 00:00 0
         [vdso]
2024-06-07T07:59:00.084375+02:00 lxdvm1-x14-u19-r1 lxd.daemon[3333]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0
         [vsyscall]

Jun 07 '24 15:06 webdock-io

Hm, interesting. All 3 cases are a bit different, but root cause is the same. Invalid value of INTTYPE_TO_PTR(fi->fh).

Jun 07 '24 15:06 mihalicyn

@webdock-io can you please show uname -a? Have you recently updated your kernel?

Jun 07 '24 15:06 mihalicyn

Sure - all our instances with the issue are identical to this:

Linux lxdvm2-x14-u26-r1 6.8.0-31-generic #31-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 00:40:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Jun 07 '24 15:06 webdock-io

all our instances with the issue are identical to this:

Did this problem appear recently? After a kernel upgrade? Or after LXD upgrade? (or both?..)

Jun 07 '24 16:06 mihalicyn

Well yes and no... These are not old / upgraded systems.. they are all brand new. All our newly deployed systems are on Noble and the latest kernel and latest stable lxd version.

These are also systems where we are running containers inside VMs which we haven't done before

So this is all new and the first time we run significant number of containers and load on these new systems

We are not seeing this on any of our old systems

We did do a lot of load testing before deployment and didn't see this happen. But we may have missed it as we weren't actively looking at lxcfs, rather at system stability

Jun 07 '24 16:06 webdock-io

@webdock-io thanks for providing an extra context on this ;-)

Can you try to collect a crash dump from the LXCFS?

All you need to do is (as root):

echo '|/bin/sh -c $@ -- eval exec cat > /var/crash/core-%e.%p' > /proc/sys/kernel/core_pattern

and wait until a next crash. Then collect a crashdump file in /var/crash/ directory and share it with me. (you can GPG encrypt it if you want with my key gpg --encrypt your_core_dump --recipient [email protected])

Jun 07 '24 17:06 mihalicyn

Hello. We have now run the command on all thus-far affected systems. If it happens again in the same location(s) we will have a dump for you

Jun 08 '24 12:06 webdock-io

Small update: I'm really not sure we will see this type of crash again. We identified a config issue on our systems where our ZFS ARC cache limit was set way too low. This was causing abnormally high IOWAIT on our systems which seems to have been the root cause of our problems (we were also seeing system halts/crashes). After we upped our ARC cache limit to something more reasonable, we see really good behavior everywhere and thus far no lxcfs or system crashes.

This may point to something on the systems or in the kernel, due to the io bottleneck, getting messed up. It's only been a couple of days since we made this change, so the issue may not be completely solved - but so far it's looking good. We will be keeping the crash dump collection flag turned on and will of course revert here if we see it happening again.

Thank you for your time

Jun 10 '24 07:06 webdock-io

We identified a config issue on our systems where our ZFS ARC cache limit was set way too low. This was causing abnormally high IOWAIT on our systems which seems to have been the root cause of our problems (we were also seeing system halts/crashes).

Even when IOWAIT times are high, crashes are still crashes and we should fix it. So, you have a workaround but issue is still there. So I would not ignore it, and I would try to revert this workaround and try to get a reproduction and a crashdump so we can try to figure out what is wrong from the LXCFS/libfuse/kernel side which leads to a crashes like you've seen before.

we were also seeing system halts/crashes

so, not only LXCFS was crashing? What else was crashing then?

Jun 10 '24 09:06 mihalicyn

Hello. Yes I agree, crashes are still crashes. I think it may be difficult to get a reproducer on this, but we will have a system free soon where I can attempt to recreate the issue for you, but no guarantees.

so, not only LXCFS was crashing? What else was crashing then?

I do not want to confuse the matter. This was more of a statement that we saw other things misbehaving, namely our virtual machines. As mentioned earlier we are running these workloads inside LXD VMs - so, nested LXD if you will. Here we saw the VMs hang - no crash output, nothing in syslog no explanation that we could find, the VM would just go into a halt state and we would need to restart it to get it operational again. This did not coincide with the lxcfs crashes we saw, so they are not related per se but the root cause was the same, we think.

After we made the ARC cache happy and alleviated the bottleneck, we have not seen such a crash in 48+ hours now, where before we saw 1-2 crashes every 24 hours. So we are hopeful we have mitigated that issue as well. Although, a full VM crash is worrying to say the least, and as is the case with the lxcfs crash we haven't really fixed the issue, we've just resolved what was provoking the issue (we think/hope)

Jun 10 '24 10:06 webdock-io

Hello

We have now seen this happen on a completely new host, which had our "workaround" implemented. So this crash is still happening and is still real. However, we have had to remove your crash dump reporting flag from the hosts where we did have it activated as it was filling up our disk! We are a public cloud hosting lxd container instances, and whatever crashes were happening in customer containers were being dumped to /var/crash. Not only did this create a lot of crash dumps on some hosts, worse it was quickly filling up disk on some systems. We caught it in time and averted disaster - but we simply cannot allow this crash dumping to be active on production systems.

So... In order to try and procure a crash dump for you, we have dedicated an instance to this purpose where we are stressing it a lot. We hope this will provoke a crash sometime in the next 24-48 hours - if not, then I'm not sure what we can do as then it seems synthetic workloads will not do the trick... We will see.

Anyway, here is the latest crash on an otherwise healthy system, so you can see if it's the same (I'm thinking it is)

Crash data

2024-06-11T19:38:46.020228+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: *** signal 11
2024-06-11T19:38:46.020421+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: Register dump:
2024-06-11T19:38:46.020454+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  RAX: 00007c9e3011ef80   RBX: 000008e5000008e0   RCX: 0000008e5000008c
2024-06-11T19:38:46.020480+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  RDX: 0000000000000001   RSI: 00007c9eac104980   RDI: 00007c9e30000030
2024-06-11T19:38:46.020508+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  RBP: 00007c9e30105610   R8 : 00007c9ef721d198   R9 : 000000000000000f
2024-06-11T19:38:46.020542+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  R10: 00007c9ef721d198   R11: 0000000000000000   R12: 00007c9e30000030
2024-06-11T19:38:46.020564+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  R13: 0000000000000000   R14: 0000858330105ef0   R15: 0000000000000000
2024-06-11T19:38:46.020585+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  RSP: 00007c9eee9ffa60
2024-06-11T19:38:46.020608+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  RIP: 00007c9ef7301857   EFLAGS: 00010202
2024-06-11T19:38:46.020629+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  CS: 0033   FS: 0000   GS: 0000
2024-06-11T19:38:46.020656+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  Trap: 0000000d   Error: 00000000   OldMask: 00004007   CR2: 00000000
2024-06-11T19:38:46.020695+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-11T19:38:46.020722+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  RIP: 00000000   RDP: 00000000
2024-06-11T19:38:46.020745+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-11T19:38:46.020775+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-11T19:38:46.020804+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-11T19:38:46.020828+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-11T19:38:46.020855+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  mxcsr: 1fa0
2024-06-11T19:38:46.020899+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-11T19:38:46.020927+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-11T19:38:46.020959+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-11T19:38:46.020980+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-11T19:38:46.021002+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-11T19:38:46.021023+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-11T19:38:46.021044+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-11T19:38:46.021071+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-11T19:38:46.021097+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: Backtrace:
2024-06-11T19:38:46.021156+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /lib/x86_64-linux-gnu/libc.so.6(+0xa2857)[0x7c9ef7301857]
2024-06-11T19:38:46.021198+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /lib/x86_64-linux-gnu/libc.so.6(free+0x73)[0x7c9ef7304453]
2024-06-11T19:38:46.021220+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/liblxcfs.so(do_release_file_info+0x42)[0x7c9ef72430c8]
2024-06-11T19:38:46.021247+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/liblxcfs.so(proc_release+0x20)[0x7c9ef7238789]
2024-06-11T19:38:46.021273+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: lxcfs(+0x2f2c)[0x63e917b83f2c]
2024-06-11T19:38:46.021299+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: lxcfs(+0x3c7f)[0x63e917b84c7f]
2024-06-11T19:38:46.021345+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0xbb6a)[0x7c9ef74b6b6a]
2024-06-11T19:38:46.021372+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x104fa)[0x7c9ef74bb4fa]
2024-06-11T19:38:46.021403+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x11738)[0x7c9ef74bc738]
2024-06-11T19:38:46.021425+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x7c9ef74c913f]
2024-06-11T19:38:46.021450+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x7c9ef74c17a7]
2024-06-11T19:38:46.021476+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7c9ef72f3ac3]
2024-06-11T19:38:46.021504+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7c9ef7385850]
2024-06-11T19:38:46.021525+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: Memory map:
2024-06-11T19:38:46.021551+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 63e917b81000-63e917b89000 r-xp 00000000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-11T19:38:46.021578+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 63e917b89000-63e917b8a000 r--p 00007000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-11T19:38:46.021604+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 63e917b8a000-63e917b8b000 rw-p 00008000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-11T19:38:46.021630+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 63e91820d000-63e91825c000 rw-p 00000000 00:00 0                          [heap]
2024-06-11T19:38:46.021658+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d64000000-7c9d64203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021687+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d64203000-7c9d68000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.021709+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d68000000-7c9d68223000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021739+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d68223000-7c9d6c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.021759+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d70000000-7c9d70175000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021784+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d70175000-7c9d74000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.021805+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d74000000-7c9d74204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021826+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d74204000-7c9d78000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.021856+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d7c000000-7c9d7c229000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021877+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d7c229000-7c9d80000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.021907+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d80000000-7c9d80204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021929+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d80204000-7c9d84000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.021955+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d88000000-7c9d88215000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.021984+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d88215000-7c9d8c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.022006+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d8c000000-7c9d8c172000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.022029+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d8c172000-7c9d90000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.022073+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d94000000-7c9d94119000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.022099+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d94119000-7c9d98000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026498+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d98000000-7c9d9816f000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026549+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9d9816f000-7c9d9c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026579+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9da0000000-7c9da016d000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026606+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9da016d000-7c9da4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026629+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9da4000000-7c9da4204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026652+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9da4204000-7c9da8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026681+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dac000000-7c9dac177000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026703+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dac177000-7c9db0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026734+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9db0000000-7c9db0203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026768+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9db0203000-7c9db4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026797+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9db8000000-7c9db8168000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026819+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9db8168000-7c9dbc000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026850+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dbc000000-7c9dbc204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026876+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dbc204000-7c9dc0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026903+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dc4000000-7c9dc4204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026930+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dc4204000-7c9dc8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.026952+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dc8000000-7c9dc8177000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.026974+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dc8177000-7c9dcc000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027000+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dd0000000-7c9dd0113000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027022+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dd0113000-7c9dd4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027048+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dd4000000-7c9dd416e000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027073+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dd416e000-7c9dd8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027095+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ddc000000-7c9ddc204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027122+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ddc204000-7c9de0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027143+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9de0000000-7c9de010f000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027169+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9de010f000-7c9de4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027207+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9de8000000-7c9de8203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027230+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9de8203000-7c9dec000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027257+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dec000000-7c9dec179000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027279+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9dec179000-7c9df0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027320+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9df4000000-7c9df4203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027348+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9df4203000-7c9df8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027374+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9df8000000-7c9df8114000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027399+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9df8114000-7c9dfc000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027420+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e00000000-7c9e00213000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027448+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e00213000-7c9e04000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027474+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e04000000-7c9e04203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027504+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e04203000-7c9e08000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027534+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e0c000000-7c9e0c209000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027591+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e0c209000-7c9e10000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027617+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e10000000-7c9e10203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027640+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e10203000-7c9e14000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027666+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e18000000-7c9e1816b000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027693+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e1816b000-7c9e1c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027716+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e1c000000-7c9e1c174000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027742+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e1c174000-7c9e20000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027769+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e24000000-7c9e24173000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027795+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e24173000-7c9e28000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027814+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e28000000-7c9e28203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027869+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e28203000-7c9e2c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027896+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e30000000-7c9e30204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027918+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e30204000-7c9e34000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027944+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e34000000-7c9e34145000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.027967+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e34145000-7c9e38000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.027993+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e3c000000-7c9e3c204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.028020+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e3c204000-7c9e40000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.028046+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e40000000-7c9e40174000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.028068+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e40174000-7c9e44000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.028090+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e48000000-7c9e48113000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.028111+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e48113000-7c9e4c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.028138+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e4c000000-7c9e4c17e000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.028165+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e4c17e000-7c9e50000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030279+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e54000000-7c9e5416c000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030350+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e5416c000-7c9e58000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030375+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e58000000-7c9e58164000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030402+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e58164000-7c9e5c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030423+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e60000000-7c9e6016d000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030468+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e6016d000-7c9e64000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030491+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e64000000-7c9e64203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030514+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e64203000-7c9e68000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030541+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e6c000000-7c9e6c205000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030571+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e6c205000-7c9e70000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030592+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e70000000-7c9e70172000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030627+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e70172000-7c9e74000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030658+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e78000000-7c9e78113000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030693+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e78113000-7c9e7c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030715+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e7c000000-7c9e7c171000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030738+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e7c171000-7c9e80000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030765+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e84000000-7c9e84205000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030786+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e84205000-7c9e88000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030847+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e88000000-7c9e88203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030878+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e88203000-7c9e8c000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030899+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e90000000-7c9e90203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030926+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e90203000-7c9e94000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030947+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e94000000-7c9e94204000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.030969+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e94204000-7c9e98000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.030996+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e9c000000-7c9e9c235000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031019+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9e9c235000-7c9ea0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031041+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ea0000000-7c9ea0180000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031066+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ea0180000-7c9ea4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031097+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ea8000000-7c9ea817c000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031124+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ea817c000-7c9eac000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031150+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eac000000-7c9eac179000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031172+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eac179000-7c9eb0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031207+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eb4000000-7c9eb4179000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031231+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eb4179000-7c9eb8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031253+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eb8000000-7c9eb8061000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031278+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eb8061000-7c9ebc000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031300+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ec0000000-7c9ec0236000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031338+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ec0236000-7c9ec4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031360+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ec4000000-7c9ec4179000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031390+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ec4179000-7c9ec8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031423+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ecc000000-7c9ecc16b000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031454+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ecc16b000-7c9ed0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031482+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ed0000000-7c9ed016b000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031505+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ed016b000-7c9ed4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031531+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ed7600000-7c9ed7601000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031558+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ed7601000-7c9ed7e01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031580+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ed8000000-7c9ed8203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031602+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ed8203000-7c9edc000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031624+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9edc000000-7c9edc17b000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031646+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9edc17b000-7c9ee0000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031676+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee0400000-7c9ee0401000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031700+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee0401000-7c9ee0c01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031721+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee0e00000-7c9ee0e01000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031747+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee0e01000-7c9ee1601000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031768+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee1800000-7c9ee1801000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031790+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee1801000-7c9ee2001000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031816+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee2200000-7c9ee2201000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031838+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee2201000-7c9ee2a01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031860+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee2c00000-7c9ee2c01000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031901+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee2c01000-7c9ee3401000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031933+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee3600000-7c9ee3601000 ---p 00000000 00:00 0
2024-06-11T19:38:46.031959+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee3601000-7c9ee3e01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.031985+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee4000000-7c9ee4177000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032004+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee4177000-7c9ee8000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032026+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee8000000-7c9ee8203000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032052+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ee8203000-7c9eec000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032082+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eed800000-7c9eed801000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032109+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eed801000-7c9eee001000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032135+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eee200000-7c9eee201000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032157+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eee201000-7c9eeea01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032198+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eeec00000-7c9eeec01000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032225+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eeec01000-7c9eef401000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032249+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eef600000-7c9eef601000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032280+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9eef601000-7c9eefe01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032326+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef0000000-7c9ef0172000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032352+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef0172000-7c9ef4000000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032374+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef4c00000-7c9ef4c01000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032397+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef4c01000-7c9ef5401000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032423+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef5600000-7c9ef5601000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032450+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef5601000-7c9ef5e01000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032473+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef6000000-7c9ef6001000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032495+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef6001000-7c9ef6801000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032522+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef699e000-7c9ef6a00000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032578+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef6a00000-7c9ef6a01000 ---p 00000000 00:00 0
2024-06-11T19:38:46.032601+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef6a01000-7c9ef7201000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032623+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7201000-7c9ef721c000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032645+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef721c000-7c9ef724f000 r-xp 00000000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-11T19:38:46.032669+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef724f000-7c9ef7250000 r--p 00032000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-11T19:38:46.032694+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7250000-7c9ef7251000 rw-p 00033000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-11T19:38:46.032716+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7251000-7c9ef725f000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032738+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef725f000-7c9ef7287000 r--p 00000000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-11T19:38:46.032766+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7287000-7c9ef741c000 r-xp 00028000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-11T19:38:46.032787+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef741c000-7c9ef7474000 r--p 001bd000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-11T19:38:46.032831+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7474000-7c9ef7475000 ---p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-11T19:38:46.032858+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7475000-7c9ef7479000 r--p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-11T19:38:46.032884+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7479000-7c9ef747b000 rw-p 00219000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-11T19:38:46.032907+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef747b000-7c9ef7488000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.032933+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef7488000-7c9ef748b000 r--p 00000000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-11T19:38:46.032957+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef748b000-7c9ef74a2000 r-xp 00003000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-11T19:38:46.032984+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74a2000-7c9ef74a6000 r--p 0001a000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-11T19:38:46.033006+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74a6000-7c9ef74a7000 r--p 0001d000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-11T19:38:46.033033+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74a7000-7c9ef74a8000 rw-p 0001e000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-11T19:38:46.033057+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74a8000-7c9ef74ab000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.033079+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74ab000-7c9ef74b2000 r--p 00000000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-11T19:38:46.033109+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74b2000-7c9ef74cd000 r-xp 00007000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-11T19:38:46.033132+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74cd000-7c9ef74d7000 r--p 00022000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-11T19:38:46.033154+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74d7000-7c9ef74e9000 r--p 0002b000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-11T19:38:46.033188+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74e9000-7c9ef74ea000 rw-p 0003d000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-11T19:38:46.033217+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74ea000-7c9ef74eb000 r--p 00000000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-11T19:38:46.033241+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74eb000-7c9ef74ee000 r-xp 00001000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-11T19:38:46.033263+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74ee000-7c9ef74ef000 r--p 00004000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-11T19:38:46.033317+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74ef000-7c9ef74f0000 r--p 00005000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-11T19:38:46.033347+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74f0000-7c9ef74f1000 rw-p 00006000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-11T19:38:46.033370+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74f1000-7c9ef74f3000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.033396+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74f3000-7c9ef74f5000 r--p 00000000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-11T19:38:46.033418+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef74f5000-7c9ef751f000 r-xp 00002000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-11T19:38:46.033441+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef751f000-7c9ef752a000 r--p 0002c000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-11T19:38:46.033463+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef752a000-7c9ef752b000 rw-p 00000000 00:00 0
2024-06-11T19:38:46.033484+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef752b000-7c9ef752d000 r--p 00037000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-11T19:38:46.033504+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7c9ef752d000-7c9ef752f000 rw-p 00039000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-11T19:38:46.033526+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7ffe895f7000-7ffe89618000 rw-p 00000000 00:00 0                          [stack]
2024-06-11T19:38:46.033549+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7ffe89658000-7ffe8965c000 r--p 00000000 00:00 0                          [vvar]
2024-06-11T19:38:46.033571+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: 7ffe8965c000-7ffe8965e000 r-xp 00000000 00:00 0                          [vdso]
2024-06-11T19:38:46.033593+02:00 lxdvm2-x14-u16-r1 lxd.daemon[3114]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

Jun 12 '24 08:06 webdock-io

@mihalicyn Here is another case for you. I dont want to be reporting the same crash over and over, but I'm reporting this one as it's different somehow. So, we have "detection" of crashes which just consists of us checking if the lxcfs process is running with pgrep lxcfs. Now, we were made aware by a customer something was wrong just now, and checking we see lxcfs is not functioning properly and we do see the below crash in the syslog. However, we not only see lxcfs running in our process list, but we have two instances running. I have no idea if that is something that is possible/expected to happen and I find it strange lxcfs hasn't completely died/gone away like with the other crashes we've seen. This is what ps aux shows:

$ ps aux | grep lxcfs
root     2142417  0.0  0.0   7076  2016 pts/1    S+   13:52   0:00 grep --color=auto lxcfs
root     2599964  0.2  0.0 451664  2304 ?        Sl   Jun11   3:43 lxcfs --enable-loadavg /var/snap/lxd/common/var/lib/lxcfs -p /var/snap/lxd/common/lxcfs.pid
root     2733049  0.6  0.0 675464  2592 ?        Sl   Jun11   8:48 lxcfs --enable-loadavg /var/snap/lxd/common/var/lib/lxcfs -p /var/snap/lxd/common/lxcfs.pid

In my mind, very odd behavior in this case. We checked on further hosts, and we actually have one more host which has two instances of lxcfs running, but there it seems OK and doing its job. We are not sure what to make of this. Here is the relevant crash dump:

Crash data

2024-06-12T05:53:38.366230+02:00 lxdvm1-x14-u15-r1 kernel: tasks_rcu_exit_srcu_stall: rcu_tasks grace period number 669 (since boot) gp_state: RTGS_POST_SCAN_TASKLIST is 21156334 jiffies old.
2024-06-12T05:53:38.366269+02:00 lxdvm1-x14-u15-r1 kernel: Please check any exiting tasks stuck between calls to exit_tasks_rcu_start() and exit_tasks_rcu_finish()
2024-06-12T05:53:38.475974+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: *** signal 11
2024-06-12T05:53:38.476325+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: Register dump:
2024-06-12T05:53:38.476400+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  RAX: 0000788497de0300   RBX: 0000000000000000   RCX: 0000000000000100
2024-06-12T05:53:38.476464+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  RDX: 0000000000001687   RSI: 0000000000000100   RDI: 00007884833ff540
2024-06-12T05:53:38.476530+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  RBP: 0000000000000100   R8 : 0000000000000000   R9 : 00007884833ff630
2024-06-12T05:53:38.476605+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  R10: 0000788497d87ac0   R11: 0000788497d883c0   R12: 0000000000001687
2024-06-12T05:53:38.476679+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  R13: 00007884833ff540   R14: 0000788497bb2030   R15: 00007884833ff6c0
2024-06-12T05:53:38.476739+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  RSP: 00007884833ff530
2024-06-12T05:53:38.476799+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  RIP: 0000788497c51481   EFLAGS: 00010202
2024-06-12T05:53:38.476859+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  CS: 0033   FS: 0000   GS: 0000
2024-06-12T05:53:38.476926+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  Trap: 0000000e   Error: 00000006   OldMask: 00004007   CR2: 00000100
2024-06-12T05:53:38.476989+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  FPUCW: 0000037f   FPUSW: 00000000   TAG: 00000000
2024-06-12T05:53:38.477070+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  RIP: 00000000   RDP: 00000000
2024-06-12T05:53:38.477139+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000
2024-06-12T05:53:38.477199+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000
2024-06-12T05:53:38.477267+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000
2024-06-12T05:53:38.477320+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  ST(6) 0000 0000000000000000   ST(7) 0000 0000000000000000
2024-06-12T05:53:38.477370+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  mxcsr: 1fa0
2024-06-12T05:53:38.477422+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000
2024-06-12T05:53:38.477474+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000
2024-06-12T05:53:38.477537+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000
2024-06-12T05:53:38.477588+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000
2024-06-12T05:53:38.477639+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000
2024-06-12T05:53:38.477711+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000
2024-06-12T05:53:38.477769+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000
2024-06-12T05:53:38.477821+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]:  XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000
2024-06-12T05:53:38.477882+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: Backtrace:
2024-06-12T05:53:38.478000+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /lib/x86_64-linux-gnu/libc.so.6(+0x88481)[0x788497c51481]
2024-06-12T05:53:38.478097+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /lib/x86_64-linux-gnu/libc.so.6(snprintf+0x96)[0x788497c29856]
2024-06-12T05:53:38.478156+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/liblxcfs.so(+0x1facc)[0x788497ba5acc]
2024-06-12T05:53:38.478219+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/liblxcfs.so(proc_read+0x17d)[0x788497ba8478]
2024-06-12T05:53:38.478297+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: lxcfs(+0x22c3)[0x615ab2cfd2c3]
2024-06-12T05:53:38.478357+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: lxcfs(+0x3a56)[0x615ab2cfea56]
2024-06-12T05:53:38.478437+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(fuse_fs_read_buf+0x1be)[0x788497e22f2e]
2024-06-12T05:53:38.478497+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x10d66)[0x788497e25d66]
2024-06-12T05:53:38.478557+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x114fa)[0x788497e264fa]
2024-06-12T05:53:38.478616+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x1e13f)[0x788497e3313f]
2024-06-12T05:53:38.478668+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /snap/lxd/current/lib/x86_64-linux-gnu/libfuse3.so.3(+0x167a7)[0x788497e2b7a7]
2024-06-12T05:53:38.478720+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x788497c5dac3]
2024-06-12T05:53:38.478790+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: /lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x788497cef850]
2024-06-12T05:53:38.478860+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: Memory map:
2024-06-12T05:53:38.478920+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 615ab2cfb000-615ab2d03000 r-xp 00000000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-12T05:53:38.478974+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 615ab2d03000-615ab2d04000 r--p 00007000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-12T05:53:38.479057+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 615ab2d04000-615ab2d05000 rw-p 00008000 07:02 44                         /snap/lxd/28463/bin/lxcfs
2024-06-12T05:53:38.479157+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 615ab4b83000-615ab4bd2000 rw-p 00000000 00:00 0                          [heap]
2024-06-12T05:53:38.479245+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78835c000000-78835c185000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.479321+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78835c185000-788360000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.479407+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788364000000-788364182000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.479481+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788364182000-788368000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.479536+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788368000000-788368188000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.479601+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788368188000-78836c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.479657+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788370000000-788370189000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.479722+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788370189000-788374000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.479800+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788374000000-788374112000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.479874+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788374112000-788378000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.479947+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78837c000000-78837c184000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.480022+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78837c184000-788380000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.480109+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788380000000-78838017e000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.480172+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78838017e000-788384000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.480228+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788388000000-788388112000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.480300+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788388112000-78838c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.480392+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78838c000000-78838c203000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.480460+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78838c203000-788390000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.480538+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788394000000-788394184000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.480949+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788394184000-788398000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.481084+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788398000000-788398177000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.481193+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788398177000-78839c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.481369+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883a0000000-7883a0203000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.481452+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883a0203000-7883a4000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.481529+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883a4000000-7883a4203000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.481621+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883a4203000-7883a8000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.481679+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883ac000000-7883ac180000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.481753+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883ac180000-7883b0000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.481818+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883b0000000-7883b0184000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.481882+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883b0184000-7883b4000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.481947+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883b8000000-7883b816a000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482011+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883b816a000-7883bc000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482110+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883bc000000-7883bc17e000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482184+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883bc17e000-7883c0000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482269+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883c4000000-7883c4176000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482337+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883c4176000-7883c8000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482403+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883c8000000-7883c8183000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482477+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883c8183000-7883cc000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482551+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883d0000000-7883d0183000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482607+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883d0183000-7883d4000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482677+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883d4000000-7883d418a000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482724+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883d418a000-7883d8000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482780+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883dc000000-7883dc203000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482837+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883dc203000-7883e0000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.482883+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883e0000000-7883e018b000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.482949+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883e018b000-7883e4000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483024+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883e8000000-7883e8191000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.483111+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883e8191000-7883ec000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483184+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883ec000000-7883ec16d000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.483267+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883ec16d000-7883f0000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483367+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883f4000000-7883f4187000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.483432+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883f4187000-7883f8000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483496+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883f8000000-7883f8175000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.483561+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7883f8175000-7883fc000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483650+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788400000000-788400187000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.483719+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788400187000-788404000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483805+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788404000000-788404188000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.483913+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788404188000-788408000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.483990+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78840c000000-78840c16e000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484075+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78840c16e000-788410000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.484147+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788410000000-78841018b000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484212+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78841018b000-788414000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.484297+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788418000000-78841816d000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484363+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78841816d000-78841c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.484428+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78841c000000-78841c189000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484492+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78841c189000-788420000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.484556+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788424000000-788424174000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484629+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788424174000-788428000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.484686+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788428000000-78842818a000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484750+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78842818a000-78842c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.484815+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788430000000-788430182000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.484879+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788430182000-788434000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.485513+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788434000000-788434203000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.485611+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788434203000-788438000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.485659+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78843c000000-78843c18a000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.485728+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78843c18a000-788440000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.485805+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788440000000-788440187000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.485892+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788440187000-788444000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.485957+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788448000000-78844817b000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486021+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78844817b000-78844c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486115+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78844c000000-78844c186000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486187+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78844c186000-788450000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486273+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788454000000-788454183000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486339+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788454183000-788458000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486404+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788458000000-78845818c000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486459+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78845818c000-78845c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486524+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788460000000-78846017c000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486579+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78846017c000-788464000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486634+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788464000000-788464178000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486700+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788464178000-788468000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486769+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78846b600000-78846b601000 ---p 00000000 00:00 0
2024-06-12T05:53:38.486834+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78846b601000-78846be01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486903+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78846c000000-78846c198000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.486958+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78846c198000-788470000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487014+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788470000000-78847018c000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.487095+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78847018c000-788474000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487169+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788476200000-788476201000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487254+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788476201000-788476a01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.487363+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788478000000-78847818a000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.487442+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78847818a000-78847c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487508+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78847c000000-78847c068000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.487583+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78847c068000-788480000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487639+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788481800000-788481801000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487695+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788481801000-788482001000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.487761+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788482200000-788482201000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487817+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788482201000-788482a01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.487888+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788482c00000-788482c01000 ---p 00000000 00:00 0
2024-06-12T05:53:38.487936+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788482c01000-788483401000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.488001+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788483600000-788483601000 ---p 00000000 00:00 0
2024-06-12T05:53:38.488089+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788483601000-788483e01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.488475+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788484000000-78848418b000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.488564+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848418b000-788488000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.488642+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788488000000-788488174000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.488700+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788488174000-78848c000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.488761+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848c400000-78848c401000 ---p 00000000 00:00 0
2024-06-12T05:53:38.488826+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848c401000-78848cc01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.488891+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848ce00000-78848ce01000 ---p 00000000 00:00 0
2024-06-12T05:53:38.488956+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848ce01000-78848d601000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489000+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848e200000-78848e201000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489026+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848e201000-78848ea01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489049+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848ec00000-78848ec01000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489097+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848ec01000-78848f401000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489125+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848f600000-78848f601000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489152+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78848f601000-78848fe01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489178+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788490000000-78849018b000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489201+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 78849018b000-788494000000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489224+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788494a00000-788494a01000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489264+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788494a01000-788495201000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489284+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788495400000-788495401000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489311+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788495401000-788495c01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489337+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788496800000-788496801000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489364+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788496801000-788497001000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489395+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497200000-788497201000 ---p 00000000 00:00 0
2024-06-12T05:53:38.489422+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497201000-788497a01000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489448+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497a82000-788497a84000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489468+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497b0b000-788497b86000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489500+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497b86000-788497bb9000 r-xp 00000000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-12T05:53:38.489547+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497bb9000-788497bba000 r--p 00032000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-12T05:53:38.489575+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497bba000-788497bbb000 rw-p 00033000 07:02 155                        /snap/lxd/28463/lib/liblxcfs.so
2024-06-12T05:53:38.489595+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497bbb000-788497bc9000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489614+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497bc9000-788497bf1000 r--p 00000000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-12T05:53:38.489637+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497bf1000-788497d86000 r-xp 00028000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-12T05:53:38.489664+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497d86000-788497dde000 r--p 001bd000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-12T05:53:38.489691+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497dde000-788497ddf000 ---p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-12T05:53:38.489718+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497ddf000-788497de3000 r--p 00215000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-12T05:53:38.489745+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497de3000-788497de5000 rw-p 00219000 07:00 7471                       /usr/lib/x86_64-linux-gnu/libc.so.6
2024-06-12T05:53:38.489772+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497de5000-788497df2000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489804+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497df2000-788497df5000 r--p 00000000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-12T05:53:38.489841+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497df5000-788497e0c000 r-xp 00003000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-12T05:53:38.489869+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e0c000-788497e10000 r--p 0001a000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-12T05:53:38.489896+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e10000-788497e11000 r--p 0001d000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-12T05:53:38.489919+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e11000-788497e12000 rw-p 0001e000 07:00 7521                       /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
2024-06-12T05:53:38.489942+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e12000-788497e15000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.489968+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e15000-788497e1c000 r--p 00000000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-12T05:53:38.489996+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e1c000-788497e37000 r-xp 00007000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-12T05:53:38.490019+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e37000-788497e41000 r--p 00022000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-12T05:53:38.490076+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e41000-788497e53000 r--p 0002b000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-12T05:53:38.490108+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e53000-788497e54000 rw-p 0003d000 07:02 565                        /snap/lxd/28463/lib/x86_64-linux-gnu/libfuse3.so.3.10.5
2024-06-12T05:53:38.490136+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e54000-788497e55000 r--p 00000000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-12T05:53:38.490163+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e55000-788497e58000 r-xp 00001000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-12T05:53:38.490273+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e58000-788497e59000 r--p 00004000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-12T05:53:38.490301+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e59000-788497e5a000 r--p 00005000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-12T05:53:38.490333+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e5a000-788497e5b000 rw-p 00006000 07:02 538                        /snap/lxd/28463/lib/x86_64-linux-gnu/libSegFault.so
2024-06-12T05:53:38.490357+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e5b000-788497e5d000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.490384+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e5d000-788497e5f000 r--p 00000000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-12T05:53:38.490411+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e5f000-788497e89000 r-xp 00002000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-12T05:53:38.490438+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e89000-788497e94000 r--p 0002c000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-12T05:53:38.490465+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e94000-788497e95000 rw-p 00000000 00:00 0
2024-06-12T05:53:38.490485+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e95000-788497e97000 r--p 00037000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-12T05:53:38.490508+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 788497e97000-788497e99000 rw-p 00039000 07:00 7442                       /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
2024-06-12T05:53:38.490540+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7ffe743d3000-7ffe743f4000 rw-p 00000000 00:00 0                          [stack]
2024-06-12T05:53:38.490571+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7ffe743f5000-7ffe743f9000 r--p 00000000 00:00 0                          [vvar]
2024-06-12T05:53:38.490599+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: 7ffe743f9000-7ffe743fb000 r-xp 00000000 00:00 0                          [vdso]
2024-06-12T05:53:38.490623+02:00 lxdvm1-x14-u15-r1 lxd.daemon[2901]: ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

Jun 12 '24 11:06 webdock-io

One last addendum here @mihalicyn we are also the guys running everything under cgroupv1, because of this: https://github.com/lxc/lxcfs/issues/538

I dont know if that has any bearing on the issue, but we thought it was relevant to mention this.

Edit: We can give you access to a test environment if you want to inspect some things or test something out. Just send us your public key at [email protected] and we will give you access. Thanks very much for your time. This issue is unfortunately becoming rather painful for us, so we really hope you can find time to check this further if you can. Fingers crossed :)

Jun 12 '24 12:06 webdock-io

@webdock-io thanks for providing an extra context on this ;-)

Can you try to collect a crash dump from the LXCFS?

All you need to do is (as root):
echo '|/bin/sh -c $@ -- eval exec cat > /var/crash/core-%e.%p' > /proc/sys/kernel/core_pattern
and wait until a next crash. Then collect a crashdump file in /var/crash/ directory and share it with me. (you can GPG encrypt it if you want with my key gpg --encrypt your_core_dump --recipient [email protected])

@mihalicyn is there a way where we can limit core dumps to disk to only happen if lxcfs crashes and not other stuff? If so we would be able to enable this infrastructure-wide and catch this for sure... We tried looking around for info on this, but it's unclear how to achieve this exactly..if you know how, that would help

Jun 12 '24 14:06 webdock-io

So... In order to try and procure a crash dump for you, we have dedicated an instance to this purpose where we are stressing it a lot. We hope this will provoke a crash sometime in the next 24-48 hours - if not, then I'm not sure what we can do as then it seems synthetic workloads will not do the trick... We will see.

We have now absolutely hammered a system for more than 24 hours with high load on cpu, memory, io and sirq. We nerfed our Arc cache and had 3 container instances doing a lot of activity. And.... Nothing. lxcfs runs fine, no crash of our VM

So, we are unable to reproduce this synthetically it seems. Maybe we just need a lot more containers... An idea could be to spin up 100+ containers and have them all do... something.

@mihalicyn Maybe to help us get a reproducer here, can you tell us if the crashing codepath relates to some specific virtualization in lxcfs? For example, if the code is tied to CPU or memory or whichever specific subsystem? Because then we could focus on creating diverse workloads against just that system, instead of hammering everything all at once.

Anyway, we await your feedback here :)

Jun 13 '24 07:06 webdock-io

Hi, @webdock-io

@mihalicyn is there a way where we can limit core dumps to disk to only happen if lxcfs crashes and not other stuff? If so we would be able to enable this infrastructure-wide and catch this for sure... We tried looking around for info on this, but it's unclear how to achieve this exactly..if you know how, that would help

You can try something like that:

Create a /root/core_dump.sh file with the following content:

# cat /root/core_dump.sh
#!/bin/sh

PROC_NAME=$1
PROC_PID=$2

[ "${PROC_NAME}" != "lxcfs" ] && exit 0

cat /dev/stdin | gzip --fast > /var/crash/core-${PROC_NAME}.${PROC_PID}.gz

chmod +x /root/core_dump.sh

# save an old core_pattern setting value:
cat /proc/sys/kernel/core_pattern > /root/core_pattern.old_value.bak

# set the new core dump handler
echo '|/root/core_dump.sh %e %p' > /proc/sys/kernel/core_pattern

Jun 13 '24 10:06 mihalicyn

Edit: We can give you access to a test environment if you want to inspect some things or test something out. Just send us your public key at [email protected] and we will give you access. Thanks very much for your time. This issue is unfortunately becoming rather painful for us, so we really hope you can find time to check this further if you can. Fingers crossed :)

This is a good option. Let's try to get some core dump with the method I've described above (with filtering) and if it doesn't work then giving me an access would be a last hope weapon.

Jun 13 '24 12:06 mihalicyn

One last addendum here @mihalicyn we are also the guys running everything under cgroupv1, because of this: #538

Ugh, that's painful. Because at some point cgroupv1 become deprecated in upstream, or even earlier, in systemd and you will have no chance to stay on it. I believe we should sort out this blocker that prevents you from the migration to cgroup-v2. It's extremely important. Let's discuss this in #538

Jun 13 '24 12:06 mihalicyn

@mihalicyn Maybe to help us get a reproducer here, can you tell us if the crashing codepath relates to some specific virtualization in lxcfs? For example, if the code is tied to CPU or memory or whichever specific subsystem? Because then we could focus on creating diverse workloads against just that system, instead of hammering everything all at once.

That's a good question. Actually, I tend to believe that crash happens not because of an issue in LXCFS, but likely it's a stack corruption somewhere in libfuse or even a kernel problem. And to investigate this deeper I need to have a memory dump of the LXCFS process (ideally, a few different ones).

Jun 13 '24 12:06 mihalicyn

@webdock-io I have edited your messages a bit and put all the crash data content inside a special markup:

<details><summary>Crash data</summary>
...
</details>

this makes this issue thread a bit more readable for others :)

Jun 13 '24 14:06 mihalicyn

@mihalicyn Thank you for the markup tip

Ugh, that's painful. Because at some point cgroupv1 become deprecated in upstream, or even earlier, in systemd and you will have no chance to stay on it. I believe we should sort out this blocker that prevents you from the migration to cgroup-v2. It's extremely important. Let's discuss this in https://github.com/lxc/lxcfs/issues/538

I am very happy you see this as important, because it is! We were starting to have the discussion internally whether we should migrate over to KVM entirely and completely drop container VPS support, as this issue would be a complete blocker for proper container support in the near-future. Now that we know that this issue is being worked on, we will hold off on pressing the panic button anytime soon :)

We are unfortunately still seeing occasional crashes as you know, and had one last night. Unfortunately we had not had the time to implement your new crash dump code. This will be done today on all our hosts, and I am confident we will have one or more crash dumps for you within the next few days.

Thank you for your continued assistance and we will revert here once we have more material for you.

Jun 14 '24 08:06 webdock-io

@mihalicyn We have a crash dump for you! However...

you can GPG encrypt it if you want with my key gpg --encrypt your_core_dump --recipient [email protected]

This command looks a bit wrong and I tried a few things but I am unable to encrypt the dump for you. It looks to me you expect me to find your pubkey somewhere? I really tried my best reading the man page for gpg and looking around to see if I could find your pubkey somewhere but I am unable. Please provide more idiot-proof steps for me here, and I'll get this encrypted and placed somewhere you can grab it asap :)

Thanks!

Jun 19 '24 14:06 webdock-io

@mihalicyn After much much faffing around I finally found your gpg key at keyserver.ubuntu.com and imported it with gpg --keyserver keyserver.ubuntu.com --recv-keys 0x801a2056d5cf7ecf6679a79bf9fe8fed56c36d56

Anyway, here is the encrypted crash dump attached. I'll post further crash dumps as they come in :) I really hope this helps you find the issue! core-lxcfs.1499342.gpg.gz

Edit: Github would not accept .gpg exension for uploads, so I just renamed the file and reversed the extension so github would eat it. You should rename it to .gz.gpg before decrypting and then decompressing

Jun 20 '24 10:06 webdock-io

@mihalicyn And here is new one from another host

core-lxcfs.1147289.gpg.gz

Jun 21 '24 12:06 webdock-io

Hi @webdock-io !

After much much faffing around I finally found your gpg key at keyserver.ubuntu.com and imported it with

Sorry about not providing you with a more detailed instruction. It was an optional step to encrypt it, but you did it right!

Thanks for providing me with your coredumps, I'm on it to investigate this issue.

Jun 21 '24 15:06 mihalicyn

Hello @mihalicyn

We now have two more crash dumps for you:

core-lxcfs.2897.gpg.gz core-lxcfs.3396.gpg.gz

Any progress on this issue so far?

Jul 01 '24 12:07 webdock-io

lxcfs lxcfs copied to clipboard

LXCFS crashes, need help diagnosing

lxcfs
lxcfs copied to clipboard