bcachefs
bcachefs copied to clipboard
Published
20 hours ago •
koverstreet
koverstreet
System hang (in shrinker?) [70315f27c]
I'm seeing some whole-system lockups after indeterminate periods (ranging from minutes after boot to days). The attached dmesg is as of 70315f27c, but this has been going on for a bit.
I've got netconsole set up, so I'll give reproducing this with all the debugging enabled a spin.
Dec 7 10:01:08 192.168.1.50 3,2199,663844484,-;bcachefs (nvme0n1 inum 1610819926 offset 0): data checksum error: expected 6df5:24b8b47a5438bcbd got eb5a:72ff5403fefbb48d (type chacha20_poly1305_80)
Dec 7 10:01:08 192.168.1.50 3,2200,663844898,-;bcachefs (nvme0n1 inum 1610819926 offset 65536): data checksum error: expected 8348:55c102097dd2f63 got 8775:6ba0e277b6f0926f (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2203,666194125,-;bcachefs (nvme0n1 inum 1073741841 offset 1851392): data checksum error: expected c3aa:51587a6fd1ea2918 got a1bc:44c2d6567f8b40ef (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2204,666194142,-;bcachefs (nvme0n1 inum 1073741841 offset 1814528): data checksum error: expected 6b11:36f170377b69d312 got 4d70:5312248ced66eb67 (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2205,666194171,-;bcachefs (nvme0n1 inum 1073741841 offset 1765376): data checksum error: expected d895:c148f4db77ad8c3a got 20e7:c22eec8359e8060d (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2206,666194174,-;bcachefs (nvme0n1 inum 1073741841 offset 1798144): data checksum error: expected 3aae:5eee81f62b30791e got 21d1:16e497fae73602ba (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2207,666194187,-;bcachefs (nvme0n1 inum 1073741841 offset 1859584): data checksum error: expected 8a51:4937a54ad4cb8cdf got 754b:a2c50aea055da455 (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2208,666194202,-;bcachefs (nvme1n1 inum 1073741841 offset 1810432): data checksum error: expected a67a:6fe8d4c95b6e1cd3 got fdd0:15045eadf25cb376 (type chacha20_poly1305_80)
Dec 7 10:01:11 192.168.1.50 3,2209,666194220,-;bcachefs (nvme0n1 inum 1073741841 offset 1830912): data checksum error: expected 621c:9c3a486db4306fe2 got 45fc:b8116ee70e9a3bd9 (type chacha20_poly1305_80)
Dec 7 10:01:17 192.168.1.50 3,2211,672555433,-;bcachefs (nvme0n1 inum 1879152719 offset 0): data checksum error: expected 2e30:d3a42076ee27c29c got e28d:c9fd5ec5a012d0ec (type chacha20_poly1305_80)
Dec 7 10:01:17 192.168.1.50 3,2212,672651577,-;bcachefs (nvme0n1 inum 1342706554 offset 0): data checksum error: expected 8619:e2f48a8d4b55d01d got ff3f:fb720bb1504b4738 (type chacha20_poly1305_80)
Dec 7 10:01:17 192.168.1.50 3,2213,672715149,-;bcachefs (nvme0n1 inum 1074074016 offset 0): data checksum error: expected d5b5:bcde5c4f60248cb7 got 7957:ddd510e2f467cee0 (type chacha20_poly1305_80)
Dec 7 10:01:17 192.168.1.50 3,2214,672747325,-;bcachefs (nvme0n1 inum 1611076118 offset 0): data checksum error: expected f429:1a378b9e11a92489 got 3426:fc5d51586a9b5f81 (type chacha20_poly1305_80)
Dec 7 10:01:17 192.168.1.50 3,2215,672774606,-;bcachefs (nvme0n1 inum 805306530 offset 32768): data checksum error: expected 56a7:fa083283fcf6121f got f007:141d6b3afda0665 (type chacha20_poly1305_80)
Dec 7 10:01:17 192.168.1.50 3,2216,672802882,-;bcachefs (nvme0n1 inum 805306530 offset 2916352): data checksum error: expected 4ae9:ac9cef70260239a got 79d7:9531b5a16127d3be (type chacha20_poly1305_80)
Dec 7 10:01:18 192.168.1.50 3,2217,673471106,-;bcachefs (nvme0n1 inum 1342178000 offset 856064): data checksum error: expected a128:55e9152dcce5b9aa got 11ff:d44446edf4cdaab0 (type chacha20_poly1305_80)
Dec 7 10:01:19 192.168.1.50 3,2218,674242993,-;bcachefs (nvme1n1 inum 626830 offset 0): data checksum error: expected d3d3:38b9c59ee65745ef got 6988:f39c90ae3a9cc446 (type chacha20_poly1305_80)
Dec 7 10:01:19 192.168.1.50 3,2219,674657087,-;bcachefs (nvme0n1 inum 537173582 offset 8691712): data checksum error: expected 9ac2:d1008765c27f8102 got b1b9:1f530570e36e05c4 (type chacha20_poly1305_80)
Dec 7 10:01:19 192.168.1.50 3,2220,674657087,-;bcachefs (nvme0n1 inum 537173582 offset 8626176): data checksum error: expected ab4:dc0d06ec4129618a got 799:f7c31b4011b0b46e (type chacha20_poly1305_80)
Dec 7 10:01:25 192.168.1.50 3,2225,680648317,-;bcachefs (nvme0n1 inum 536871805 offset 28672): data checksum error: expected ccff:a05da2571f93086e got 4686:18c2bc7dd2de0568 (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2226,681035310,-;bcachefs (nvme1n1 inum 537092250 offset 8876032): data checksum error: expected 7f29:4983dda28fa8842e got c2ab:850860f083a8e985 (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2228,681391980,-;bcachefs (nvme0n1 inum 769493 offset 0): data checksum error: expected 5742:1314be98e085644f got ff39:f1e3ae9ebc48e6bc (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2229,681392545,-;bcachefs (nvme0n1 inum 769493 offset 65536): data checksum error: expected 8221:87d5e44989e5d479 got 3f01:8bdc867dc8031d33 (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2230,681589126,-;bcachefs (nvme0n1 inum 805548233 offset 4689920): data checksum error: expected 5ff4:8fbf3ca49c3b473f got 41e6:dc9c4387f3838c07 (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2231,681589980,-;bcachefs (nvme0n1 inum 805548233 offset 5115904): data checksum error: expected ad72:7aebb9d4a7667826 got 697:b28955b11cc39a4a (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2232,681590412,-;bcachefs (nvme0n1 inum 805548233 offset 5554176): data checksum error: expected 6360:5cb3a04fd2a25e44 got eebf:a4e288d1bd13f399 (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2233,681591347,-;bcachefs (nvme0n1 inum 805548233 offset 5214208): data checksum error: expected da10:45b87418df7f9e00 got 5247:a4d394481284d4b (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2234,681609027,-;bcachefs (nvme0n1 inum 1074046632 offset 5341184): data checksum error: expected c4a0:e52263055f4efe38 got 5d8a:bddf8324461b853d (type chacha20_poly1305_80)
Dec 7 10:01:26 192.168.1.50 3,2235,681609394,-;bcachefs (nvme0n1 inum 1074046632 offset 8765440): data checksum error: expected 1172:746e0768ba685bc0 got e796:d63415884bfc899a (type chacha20_poly1305_80)
Dec 7 10:01:34 192.168.1.50 3,2252,689257243,-;bcachefs (nvme1n1 inum 537027229 offset 54775808): data checksum error: expected 84d7:f70388754519f82d got 51bd:a0348f913f6c9442 (type chacha20_poly1305_80)
Dec 7 10:01:34 192.168.1.50 3,2253,689447442,-;bcachefs (nvme1n1 inum 537027229 offset 50925568): data checksum error: expected 416b:feee92dbca48b564 got e028:ac1ce73a353edd79 (type chacha20_poly1305_80)
Dec 7 10:01:34 192.168.1.50 3,2254,689470112,-;bcachefs (nvme1n1 inum 537027229 offset 51208192): data checksum error: expected ca9d:c48200b974ef508 got 3bdb:5c9a832fb083fe50 (type chacha20_poly1305_80)
Dec 7 10:01:34 192.168.1.50 3,2255,689499963,-;bcachefs (nvme1n1 inum 537027229 offset 50917376): data checksum error: expected 416b:feee92dbca48b564 got e028:ac1ce73a353edd79 (type chacha20_poly1305_80)
Dec 7 10:01:34 192.168.1.50 3,2256,689534360,-;bcachefs (nvme1n1 inum 805310103 offset 154992640): data checksum error: expected 930:58e5f2e371ef53dc got de85:beb07c9c968c27 (type chacha20_poly1305_80)
Dec 7 10:01:50 192.168.1.50 3,2258,705850388,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 6 jiffies s: 653 root: 0x8/.
Dec 7 10:01:50 192.168.1.50 3,2259,705850404,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:04:11 192.168.1.50 3,2279,846689451,-;INFO: task GUsbEventThread:4510 blocked for more than 120 seconds.
Dec 7 10:04:11 192.168.1.50 3,2280,846689463,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:04:11 192.168.1.50 3,2281,846689467,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:04:11 192.168.1.50 3,2323,846689599,-;INFO: task pool-org.gnome.:4839 blocked for more than 120 seconds.
Dec 7 10:04:11 192.168.1.50 3,2324,846689603,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:04:11 192.168.1.50 3,2325,846689606,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2353,967522282,-;INFO: task systemd:1 blocked for more than 120 seconds.
Dec 7 10:06:12 192.168.1.50 3,2354,967522296,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2355,967522300,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2397,967522454,-;INFO: task systemd-journal:1301 blocked for more than 120 seconds.
Dec 7 10:06:12 192.168.1.50 3,2398,967522460,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2399,967522464,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2432,967522571,-;INFO: task kworker/1:0:1337 blocked for more than 120 seconds.
Dec 7 10:06:12 192.168.1.50 3,2433,967522575,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2434,967522579,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2453,967522648,-;INFO: task kworker/5:4:2449 blocked for more than 120 seconds.
Dec 7 10:06:12 192.168.1.50 3,2454,967522652,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2455,967522656,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2474,967522732,-;INFO: task GUsbEventThread:4510 blocked for more than 241 seconds.
Dec 7 10:06:12 192.168.1.50 3,2475,967522736,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2476,967522740,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2518,967522874,-;INFO: task systemd:4577 blocked for more than 120 seconds.
Dec 7 10:06:12 192.168.1.50 3,2519,967522878,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2520,967522882,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2558,967523007,-;INFO: task gnome-session-b:4721 blocked for more than 120 seconds.
Dec 7 10:06:12 192.168.1.50 3,2559,967523012,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2560,967523015,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:06:12 192.168.1.50 3,2593,967523124,-;INFO: task pool-org.gnome.:4839 blocked for more than 241 seconds.
Dec 7 10:06:12 192.168.1.50 3,2594,967523128,-; Not tainted 6.0.11+bcachefs.git20221205.70315f27c-1-generic #1-Ubuntu
Dec 7 10:06:12 192.168.1.50 3,2595,967523132,-;"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 7 10:11:56 192.168.1.50 6,2623,1310912258,-;sysrq: Keyboard mode set to system default
Dec 7 10:12:00 192.168.1.50 6,2624,1314899244,-;sysrq: Show Blocked State
Dec 7 10:12:07 192.168.1.50 6,3805,1322287234,-;sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z)
Dec 7 10:12:34 192.168.1.50 6,3806,1349544157,-;sysrq: Terminate All Tasks
Dec 7 10:12:34 192.168.1.50 7,3807,1349565861,-;rfkill: input handler enabled
Dec 7 10:12:34 192.168.1.50 3,3808,1349568953,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 4-.... } 6 jiffies s: 657 root: 0x10/.
Dec 7 10:12:34 192.168.1.50 3,3809,1349568991,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:34 192.168.1.50 6,3810,1349569007,-;Task dump for CPU 4:
Dec 7 10:12:34 192.168.1.50 6,3811,1349569018,-;task: tracker-extract state:R running task stack: 0 pid: 5581 ppid: 4153 flags:0x0000400e
Dec 7 10:12:34 192.168.1.50 6,3812,1349569044,-;Call Trace:
Dec 7 10:12:34 192.168.1.50 6,3813,1349569054,-; <TASK>
Dec 7 10:12:34 192.168.1.50 6,3814,1349569064,-; ? sched_clock+0x9/0x10
Dec 7 10:12:34 192.168.1.50 6,3815,1349569081,-; ? __smp_call_single_queue+0x44/0x60
Dec 7 10:12:34 192.168.1.50 6,3816,1349569097,-; ttwu_queue_wakelist+0x101/0x110
Dec 7 10:12:34 192.168.1.50 6,3817,1349569114,-; __cgroup_account_cputime+0x2f/0x60
Dec 7 10:12:34 192.168.1.50 6,3818,1349569129,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:34 192.168.1.50 6,3819,1349569144,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:34 192.168.1.50 6,3820,1349569163,-; ? bch2_btree_key_cache_scan+0x388/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3821,1349569214,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3822,1349569256,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:34 192.168.1.50 6,3823,1349569272,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:34 192.168.1.50 6,3824,1349569285,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:34 192.168.1.50 6,3825,1349569300,-; ? shrink_node+0x155/0x570
Dec 7 10:12:34 192.168.1.50 6,3826,1349569314,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3827,1349569331,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:34 192.168.1.50 6,3828,1349569346,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:34 192.168.1.50 6,3829,1349569362,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:34 192.168.1.50 6,3830,1349569380,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:34 192.168.1.50 6,3831,1349569394,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:34 192.168.1.50 6,3832,1349569407,-; ? policy_node+0x69/0x80
Dec 7 10:12:34 192.168.1.50 6,3833,1349569420,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:34 192.168.1.50 6,3834,1349569434,-; ? wp_page_copy+0x3ce/0x720
Dec 7 10:12:34 192.168.1.50 6,3835,1349569449,-; ? do_wp_page+0xc4/0x410
Dec 7 10:12:34 192.168.1.50 6,3836,1349569462,-; ? handle_pte_fault+0x21b/0x240
Dec 7 10:12:34 192.168.1.50 6,3837,1349569474,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:34 192.168.1.50 6,3838,1349569490,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3839,1349569502,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:34 192.168.1.50 6,3840,1349569517,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:34 192.168.1.50 6,3841,1349569533,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:34 192.168.1.50 6,3842,1349569547,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:34 192.168.1.50 6,3843,1349569559,-; </TASK>
Dec 7 10:12:34 192.168.1.50 3,3844,1349608951,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 6-.... } 6 jiffies s: 661 root: 0x40/.
Dec 7 10:12:34 192.168.1.50 3,3845,1349609001,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:34 192.168.1.50 6,3846,1349609014,-;Task dump for CPU 6:
Dec 7 10:12:34 192.168.1.50 6,3847,1349609024,-;task: kswapd0 state:R running task stack: 0 pid: 86 ppid: 2 flags:0x00004008
Dec 7 10:12:34 192.168.1.50 6,3848,1349609044,-;Call Trace:
Dec 7 10:12:34 192.168.1.50 6,3849,1349609051,-; <TASK>
Dec 7 10:12:34 192.168.1.50 6,3850,1349609059,-; ? raw_spin_rq_unlock+0x10/0x40
Dec 7 10:12:34 192.168.1.50 6,3851,1349609072,-; __schedule+0x472/0x570
Dec 7 10:12:34 192.168.1.50 6,3852,1349609083,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:34 192.168.1.50 6,3853,1349609095,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:34 192.168.1.50 6,3854,1349609110,-; ? bch2_btree_key_cache_scan+0x3a1/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3855,1349609149,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3856,1349609180,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:34 192.168.1.50 6,3857,1349609193,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:34 192.168.1.50 6,3858,1349609203,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:34 192.168.1.50 6,3859,1349609215,-; ? shrink_node+0x155/0x570
Dec 7 10:12:34 192.168.1.50 6,3860,1349609225,-; ? balance_pgdat+0x7a0/0x7a0
Dec 7 10:12:34 192.168.1.50 6,3861,1349609236,-; ? balance_pgdat+0x367/0x7a0
Dec 7 10:12:34 192.168.1.50 6,3862,1349609248,-; ? balance_pgdat+0x7a0/0x7a0
Dec 7 10:12:34 192.168.1.50 6,3863,1349609259,-; ? kswapd+0x109/0x1d0
Dec 7 10:12:34 192.168.1.50 6,3864,1349609268,-; ? balance_pgdat+0x7a0/0x7a0
Dec 7 10:12:34 192.168.1.50 6,3865,1349609278,-; ? kthread+0xe6/0x110
Dec 7 10:12:34 192.168.1.50 6,3866,1349609289,-; ? kthread_complete_and_exit+0x20/0x20
Dec 7 10:12:34 192.168.1.50 6,3867,1349609301,-; ? ret_from_fork+0x1f/0x30
Dec 7 10:12:34 192.168.1.50 6,3868,1349609313,-; </TASK>
Dec 7 10:12:34 192.168.1.50 3,3869,1349696989,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 6 jiffies s: 665 root: 0x4/.
Dec 7 10:12:34 192.168.1.50 3,3870,1349697023,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:34 192.168.1.50 6,3871,1349697053,-;Task dump for CPU 2:
Dec 7 10:12:34 192.168.1.50 6,3872,1349697062,-;task: gnome-shell state:R running task stack: 0 pid: 4736 ppid: 4577 flags:0x0000400e
Dec 7 10:12:34 192.168.1.50 6,3873,1349697082,-;Call Trace:
Dec 7 10:12:34 192.168.1.50 6,3874,1349697089,-; <TASK>
Dec 7 10:12:34 192.168.1.50 6,3875,1349697099,-; ? mutex_spin_on_owner+0x96/0xf0
Dec 7 10:12:34 192.168.1.50 6,3876,1349697113,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:34 192.168.1.50 6,3877,1349697126,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:34 192.168.1.50 6,3878,1349697139,-; ? bch2_btree_key_cache_scan+0x23e/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3879,1349697180,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:34 192.168.1.50 6,3880,1349697192,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:34 192.168.1.50 6,3881,1349697202,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:34 192.168.1.50 6,3882,1349697214,-; ? shrink_node+0x155/0x570
Dec 7 10:12:34 192.168.1.50 6,3883,1349697226,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3884,1349697238,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:34 192.168.1.50 6,3885,1349697250,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:34 192.168.1.50 6,3886,1349697261,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:34 192.168.1.50 6,3887,1349697276,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:34 192.168.1.50 6,3888,1349697287,-; ? alloc_pages+0x90/0x160
Dec 7 10:12:34 192.168.1.50 6,3889,1349697297,-; ? __get_free_pages+0x11/0x50
Dec 7 10:12:34 192.168.1.50 6,3890,1349697306,-; ? __pollwait+0x9e/0xe0
Dec 7 10:12:34 192.168.1.50 6,3891,1349697316,-; ? unix_poll+0x25/0x120
Dec 7 10:12:34 192.168.1.50 6,3892,1349697327,-; ? sock_poll+0x55/0x100
Dec 7 10:12:34 192.168.1.50 6,3893,1349697337,-; ? do_poll.constprop.0+0x12b/0x3d0
Dec 7 10:12:34 192.168.1.50 6,3894,1349697349,-; ? do_sys_poll+0x1ef/0x290
Dec 7 10:12:34 192.168.1.50 6,3895,1349697361,-; ? poll_initwait+0x50/0x50
Dec 7 10:12:34 192.168.1.50 6,3896,1349697372,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3897,1349697381,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3898,1349697390,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3899,1349697399,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3900,1349697408,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3901,1349697417,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3902,1349697427,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3903,1349697436,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3904,1349697445,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:34 192.168.1.50 6,3905,1349697454,-; ? __x64_sys_poll+0xc7/0x150
Dec 7 10:12:34 192.168.1.50 6,3906,1349697464,-; ? do_syscall_64+0x58/0x90
Dec 7 10:12:34 192.168.1.50 6,3907,1349697475,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:34 192.168.1.50 6,3908,1349697487,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:34 192.168.1.50 6,3909,1349697496,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:34 192.168.1.50 6,3910,1349697507,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:34 192.168.1.50 6,3911,1349697516,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:34 192.168.1.50 6,3912,1349697526,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:34 192.168.1.50 6,3913,1349697535,-; ? sysvec_apic_timer_interrupt+0x4b/0xd0
Dec 7 10:12:34 192.168.1.50 6,3914,1349697547,-; ? entry_SYSCALL_64_after_hwframe+0x63/0xcd
Dec 7 10:12:34 192.168.1.50 6,3915,1349697560,-; </TASK>
Dec 7 10:12:34 192.168.1.50 3,3916,1349757004,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-.... } 6 jiffies s: 669 root: 0x1/.
Dec 7 10:12:34 192.168.1.50 3,3917,1349757031,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:34 192.168.1.50 6,3918,1349757047,-;Task dump for CPU 0:
Dec 7 10:12:34 192.168.1.50 6,3919,1349757055,-;task: grpcpp_sync_ser state:R running task stack: 0 pid: 4565 ppid: 1 flags:0x00004008
Dec 7 10:12:34 192.168.1.50 6,3920,1349757074,-;Call Trace:
Dec 7 10:12:34 192.168.1.50 6,3921,1349757082,-; <TASK>
Dec 7 10:12:34 192.168.1.50 6,3922,1349757089,-; __schedule+0x215/0x570
Dec 7 10:12:34 192.168.1.50 6,3923,1349757104,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:34 192.168.1.50 6,3924,1349757116,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:34 192.168.1.50 6,3925,1349757130,-; ? bch2_btree_key_cache_scan+0x23e/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3926,1349757168,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3927,1349757198,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:34 192.168.1.50 6,3928,1349757213,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:34 192.168.1.50 6,3929,1349757223,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:34 192.168.1.50 6,3930,1349757234,-; ? shrink_node+0x155/0x570
Dec 7 10:12:34 192.168.1.50 6,3931,1349757245,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3932,1349757257,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:34 192.168.1.50 6,3933,1349757271,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:34 192.168.1.50 6,3934,1349757283,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:34 192.168.1.50 6,3935,1349757297,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:34 192.168.1.50 6,3936,1349757308,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:34 192.168.1.50 6,3937,1349757320,-; ? policy_node+0x69/0x80
Dec 7 10:12:34 192.168.1.50 6,3938,1349757330,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:34 192.168.1.50 6,3939,1349757340,-; ? do_anonymous_page+0xda/0x3a0
Dec 7 10:12:34 192.168.1.50 6,3940,1349757351,-; ? handle_pte_fault+0x20e/0x240
Dec 7 10:12:34 192.168.1.50 6,3941,1349758344,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:34 192.168.1.50 6,3942,1349759363,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3943,1349760369,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:34 192.168.1.50 6,3944,1349761373,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:34 192.168.1.50 6,3945,1349762365,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:34 192.168.1.50 6,3946,1349763349,-; </TASK>
Dec 7 10:12:34 192.168.1.50 3,3947,1349820946,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 6 jiffies s: 673 root: 0x8/.
Dec 7 10:12:34 192.168.1.50 3,3948,1349821949,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:34 192.168.1.50 6,3949,1349822946,-;Task dump for CPU 3:
Dec 7 10:12:34 192.168.1.50 6,3950,1349823941,-;task: grpcpp_sync_ser state:R running task stack: 0 pid: 4565 ppid: 1 flags:0x00004008
Dec 7 10:12:34 192.168.1.50 6,3951,1349824962,-;Call Trace:
Dec 7 10:12:34 192.168.1.50 6,3952,1349825999,-; <TASK>
Dec 7 10:12:34 192.168.1.50 6,3953,1349827035,-; ? bch2_btree_key_cache_scan+0x24a/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3954,1349828103,-; bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:34 192.168.1.50 6,3955,1349829171,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:34 192.168.1.50 6,3956,1349830621,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:34 192.168.1.50 6,3957,1349831665,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:34 192.168.1.50 6,3958,1349832709,-; ? shrink_node+0x155/0x570
Dec 7 10:12:34 192.168.1.50 6,3959,1349833742,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3960,1349834767,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:34 192.168.1.50 6,3961,1349835792,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:34 192.168.1.50 6,3962,1349836817,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:34 192.168.1.50 6,3963,1349837849,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:34 192.168.1.50 6,3964,1349838871,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:34 192.168.1.50 6,3965,1349839893,-; ? policy_node+0x69/0x80
Dec 7 10:12:34 192.168.1.50 6,3966,1349840916,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:34 192.168.1.50 6,3967,1349841946,-; ? do_anonymous_page+0xda/0x3a0
Dec 7 10:12:34 192.168.1.50 6,3968,1349842971,-; ? handle_pte_fault+0x20e/0x240
Dec 7 10:12:34 192.168.1.50 6,3969,1349843993,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:34 192.168.1.50 6,3970,1349845026,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:34 192.168.1.50 6,3971,1349846051,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:34 192.168.1.50 6,3972,1349847085,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:34 192.168.1.50 6,3973,1349848110,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:34 192.168.1.50 6,3974,1349849141,-; </TASK>
Dec 7 10:12:35 192.168.1.50 3,3975,1349888985,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 6-.... } 7 jiffies s: 677 root: 0x40/.
Dec 7 10:12:35 192.168.1.50 3,3976,1349890056,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:35 192.168.1.50 6,3977,1349891107,-;Task dump for CPU 6:
Dec 7 10:12:35 192.168.1.50 6,3978,1349892159,-;task: gdbus state:R running task stack: 0 pid: 4681 ppid: 4577 flags:0x0000400e
Dec 7 10:12:35 192.168.1.50 6,3979,1349893225,-;Call Trace:
Dec 7 10:12:35 192.168.1.50 6,3980,1349894271,-; <TASK>
Dec 7 10:12:35 192.168.1.50 6,3981,1349895315,-; ? mutex_spin_on_owner+0x8f/0xf0
Dec 7 10:12:35 192.168.1.50 6,3982,1349896355,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:35 192.168.1.50 6,3983,1349897381,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:35 192.168.1.50 6,3984,1349898418,-; ? bch2_btree_key_cache_scan+0x3a1/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,3985,1349899479,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,3986,1349900534,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:35 192.168.1.50 6,3987,1349901575,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:35 192.168.1.50 6,3988,1349901579,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:35 192.168.1.50 6,3989,1349901581,-; ? shrink_node+0x155/0x570
Dec 7 10:12:35 192.168.1.50 6,3990,1349901584,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:35 192.168.1.50 6,3991,1349901587,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:35 192.168.1.50 6,3992,1349901590,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:35 192.168.1.50 6,3993,1349901593,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:35 192.168.1.50 6,3994,1349901597,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:35 192.168.1.50 6,3995,1349901600,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:35 192.168.1.50 6,3996,1349901601,-; ? policy_node+0x69/0x80
Dec 7 10:12:35 192.168.1.50 6,3997,1349901604,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:35 192.168.1.50 6,3998,1349901606,-; ? do_anonymous_page+0xda/0x3a0
Dec 7 10:12:35 192.168.1.50 6,3999,1349901608,-; ? handle_pte_fault+0x20e/0x240
Dec 7 10:12:35 192.168.1.50 6,4000,1349915043,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:35 192.168.1.50 6,4001,1349916084,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4002,1349917120,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:35 192.168.1.50 6,4003,1349918162,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:35 192.168.1.50 6,4004,1349919192,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:35 192.168.1.50 6,4005,1349920213,-; </TASK>
Dec 7 10:12:35 192.168.1.50 3,4006,1349948979,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 6 jiffies s: 681 root: 0x4/.
Dec 7 10:12:35 192.168.1.50 3,4007,1349950015,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:35 192.168.1.50 6,4008,1349951037,-;Task dump for CPU 2:
Dec 7 10:12:35 192.168.1.50 6,4009,1349952062,-;task: gdbus state:R running task stack: 0 pid: 4681 ppid: 4577 flags:0x0000400e
Dec 7 10:12:35 192.168.1.50 6,4010,1349953115,-;Call Trace:
Dec 7 10:12:35 192.168.1.50 6,4011,1349954156,-; <TASK>
Dec 7 10:12:35 192.168.1.50 6,4012,1349955196,-; ? bch2_btree_key_cache_scan+0x24a/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,4013,1349956262,-; bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,4014,1349957329,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:35 192.168.1.50 6,4015,1349958377,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:35 192.168.1.50 6,4016,1349959424,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:35 192.168.1.50 6,4017,1349960472,-; ? shrink_node+0x155/0x570
Dec 7 10:12:35 192.168.1.50 6,4018,1349961522,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4019,1349962567,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:35 192.168.1.50 6,4020,1349963614,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:35 192.168.1.50 6,4021,1349964660,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:35 192.168.1.50 6,4022,1349965715,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:35 192.168.1.50 6,4023,1349966764,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:35 192.168.1.50 6,4024,1349967813,-; ? policy_node+0x69/0x80
Dec 7 10:12:35 192.168.1.50 6,4025,1349968862,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:35 192.168.1.50 6,4026,1349969914,-; ? do_anonymous_page+0xda/0x3a0
Dec 7 10:12:35 192.168.1.50 6,4027,1349970967,-; ? handle_pte_fault+0x20e/0x240
Dec 7 10:12:35 192.168.1.50 6,4028,1349972018,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:35 192.168.1.50 6,4029,1349973072,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4030,1349974125,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:35 192.168.1.50 6,4031,1349975180,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:35 192.168.1.50 6,4032,1349976235,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:35 192.168.1.50 6,4033,1349977297,-; </TASK>
Dec 7 10:12:35 192.168.1.50 3,4034,1350016983,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 7-.... } 6 jiffies s: 685 root: 0x80/.
Dec 7 10:12:35 192.168.1.50 3,4035,1350018090,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:35 192.168.1.50 6,4036,1350019184,-;Task dump for CPU 7:
Dec 7 10:12:35 192.168.1.50 6,4037,1350020283,-;task: NetworkManager state:R running task stack: 0 pid: 3024 ppid: 1 flags:0x00004006
Dec 7 10:12:35 192.168.1.50 6,4038,1350021408,-;Call Trace:
Dec 7 10:12:35 192.168.1.50 6,4039,1350022528,-; <TASK>
Dec 7 10:12:35 192.168.1.50 6,4040,1350023648,-; ? mutex_spin_on_owner+0x8f/0xf0
Dec 7 10:12:35 192.168.1.50 6,4041,1350024772,-; ? common_interrupt+0x54/0xb0
Dec 7 10:12:35 192.168.1.50 6,4042,1350025898,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:35 192.168.1.50 6,4043,1350025901,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:35 192.168.1.50 6,4044,1350028441,-; ? bch2_btree_key_cache_scan+0x39c/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,4045,1350029595,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:35 192.168.1.50 6,4046,1350030730,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:35 192.168.1.50 6,4047,1350031864,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:35 192.168.1.50 6,4048,1350033000,-; ? shrink_node+0x155/0x570
Dec 7 10:12:35 192.168.1.50 6,4049,1350034133,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4050,1350035267,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:35 192.168.1.50 6,4051,1350036392,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:35 192.168.1.50 6,4052,1350037507,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:35 192.168.1.50 6,4053,1350038620,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:35 192.168.1.50 6,4054,1350039722,-; ? alloc_pages+0x90/0x160
Dec 7 10:12:35 192.168.1.50 6,4055,1350040813,-; ? __get_free_pages+0x11/0x50
Dec 7 10:12:35 192.168.1.50 6,4056,1350041908,-; ? __pollwait+0x9e/0xe0
Dec 7 10:12:35 192.168.1.50 6,4057,1350043003,-; ? __ep_eventpoll_poll.isra.0+0x6b/0x170
Dec 7 10:12:35 192.168.1.50 6,4058,1350044101,-; ? ep_eventpoll_poll+0xe/0x20
Dec 7 10:12:35 192.168.1.50 6,4059,1350045199,-; ? do_poll.constprop.0+0x12b/0x3d0
Dec 7 10:12:35 192.168.1.50 6,4060,1350046301,-; ? do_sys_poll+0x1ef/0x290
Dec 7 10:12:35 192.168.1.50 6,4061,1350047402,-; ? poll_initwait+0x50/0x50
Dec 7 10:12:35 192.168.1.50 6,4062,1350048499,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4063,1350049601,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4064,1350050699,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4065,1350051794,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4066,1350052884,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4067,1350053975,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4068,1350055060,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4069,1350056138,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4070,1350057218,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4071,1350058292,-; ? __x64_sys_poll+0xc7/0x150
Dec 7 10:12:35 192.168.1.50 6,4072,1350059366,-; ? do_syscall_64+0x58/0x90
Dec 7 10:12:35 192.168.1.50 6,4073,1350060440,-; ? exit_to_user_mode_prepare+0x30/0xb0
Dec 7 10:12:35 192.168.1.50 6,4074,1350061522,-; ? exit_to_user_mode_prepare+0x30/0xb0
Dec 7 10:12:35 192.168.1.50 6,4075,1350062597,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:35 192.168.1.50 6,4076,1350063673,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4077,1350064747,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4078,1350065822,-; ? exit_to_user_mode_prepare+0x9b/0xb0
Dec 7 10:12:35 192.168.1.50 6,4079,1350065824,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:35 192.168.1.50 6,4080,1350068244,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4081,1350069320,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4082,1350070378,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4083,1350071424,-; ? entry_SYSCALL_64_after_hwframe+0x63/0xcd
Dec 7 10:12:35 192.168.1.50 6,4084,1350072473,-; </TASK>
Dec 7 10:12:35 192.168.1.50 3,4085,1350096980,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 7-.... } 6 jiffies s: 689 root: 0x80/.
Dec 7 10:12:35 192.168.1.50 3,4086,1350098038,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:35 192.168.1.50 6,4087,1350099090,-;Task dump for CPU 7:
Dec 7 10:12:35 192.168.1.50 6,4088,1350100142,-;task: NetworkManager state:R running task stack: 0 pid: 3024 ppid: 1 flags:0x00004006
Dec 7 10:12:35 192.168.1.50 6,4089,1350101213,-;Call Trace:
Dec 7 10:12:35 192.168.1.50 6,4090,1350102275,-; <TASK>
Dec 7 10:12:35 192.168.1.50 6,4091,1350103338,-; ? __smp_call_single_queue+0x44/0x60
Dec 7 10:12:35 192.168.1.50 6,4092,1350104408,-; ? ttwu_queue_wakelist+0x101/0x110
Dec 7 10:12:35 192.168.1.50 6,4093,1350105479,-; ? try_to_wake_up+0xea/0x5a0
Dec 7 10:12:35 192.168.1.50 6,4094,1350106545,-; ? swake_up_one+0x58/0x70
Dec 7 10:12:35 192.168.1.50 6,4095,1350107610,-; ? common_interrupt+0x54/0xb0
Dec 7 10:12:35 192.168.1.50 6,4096,1350108674,-; ? asm_common_interrupt+0x27/0x40
Dec 7 10:12:35 192.168.1.50 6,4097,1350109737,-; ? bch2_btree_key_cache_scan+0x393/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,4098,1350110825,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:35 192.168.1.50 6,4099,1350111889,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:35 192.168.1.50 6,4100,1350112953,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:35 192.168.1.50 6,4101,1350114017,-; ? shrink_node+0x155/0x570
Dec 7 10:12:35 192.168.1.50 6,4102,1350115080,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4103,1350116147,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:35 192.168.1.50 6,4104,1350117219,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:35 192.168.1.50 6,4105,1350118287,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:35 192.168.1.50 6,4106,1350119357,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:35 192.168.1.50 6,4107,1350120424,-; ? alloc_pages+0x90/0x160
Dec 7 10:12:35 192.168.1.50 6,4108,1350121489,-; ? __get_free_pages+0x11/0x50
Dec 7 10:12:35 192.168.1.50 6,4109,1350122553,-; ? __pollwait+0x9e/0xe0
Dec 7 10:12:35 192.168.1.50 6,4110,1350123605,-; ? __ep_eventpoll_poll.isra.0+0x6b/0x170
Dec 7 10:12:35 192.168.1.50 6,4111,1350124652,-; ? ep_eventpoll_poll+0xe/0x20
Dec 7 10:12:35 192.168.1.50 6,4112,1350125695,-; ? do_poll.constprop.0+0x12b/0x3d0
Dec 7 10:12:35 192.168.1.50 6,4113,1350126732,-; ? do_sys_poll+0x1ef/0x290
Dec 7 10:12:35 192.168.1.50 6,4114,1350127758,-; ? poll_initwait+0x50/0x50
Dec 7 10:12:35 192.168.1.50 6,4115,1350128785,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4116,1350129812,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4117,1350130833,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4118,1350131852,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4119,1350132864,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4120,1350133877,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4121,1350134883,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4122,1350135887,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4123,1350136889,-; ? __pollwait+0xe0/0xe0
Dec 7 10:12:35 192.168.1.50 6,4124,1350137891,-; ? __x64_sys_poll+0xc7/0x150
Dec 7 10:12:35 192.168.1.50 6,4125,1350138892,-; ? do_syscall_64+0x58/0x90
Dec 7 10:12:35 192.168.1.50 6,4126,1350139890,-; ? exit_to_user_mode_prepare+0x30/0xb0
Dec 7 10:12:35 192.168.1.50 6,4127,1350140890,-; ? exit_to_user_mode_prepare+0x30/0xb0
Dec 7 10:12:35 192.168.1.50 6,4128,1350141889,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:35 192.168.1.50 6,4129,1350142887,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4130,1350143883,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4131,1350144879,-; ? exit_to_user_mode_prepare+0x9b/0xb0
Dec 7 10:12:35 192.168.1.50 6,4132,1350145880,-; ? syscall_exit_to_user_mode+0x29/0x50
Dec 7 10:12:35 192.168.1.50 6,4133,1350145883,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4134,1350145884,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4135,1350145886,-; ? do_syscall_64+0x67/0x90
Dec 7 10:12:35 192.168.1.50 6,4136,1350145887,-; ? entry_SYSCALL_64_after_hwframe+0x63/0xcd
Dec 7 10:12:35 192.168.1.50 6,4137,1350145891,-; </TASK>
Dec 7 10:12:35 192.168.1.50 3,4138,1350172981,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 7 jiffies s: 693 root: 0x4/.
Dec 7 10:12:35 192.168.1.50 3,4139,1350174018,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:35 192.168.1.50 6,4140,1350175037,-;Task dump for CPU 2:
Dec 7 10:12:35 192.168.1.50 6,4141,1350176045,-;task: gmain state:R running task stack: 0 pid: 4837 ppid: 4577 flags:0x0000400e
Dec 7 10:12:35 192.168.1.50 6,4142,1350177068,-;Call Trace:
Dec 7 10:12:35 192.168.1.50 6,4143,1350178081,-; <TASK>
Dec 7 10:12:35 192.168.1.50 6,4144,1350179084,-; ? mutex_spin_on_owner+0x8f/0xf0
Dec 7 10:12:35 192.168.1.50 6,4145,1350180082,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:35 192.168.1.50 6,4146,1350181089,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:35 192.168.1.50 6,4147,1350182089,-; ? bch2_btree_key_cache_scan+0x23e/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,4148,1350183108,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:35 192.168.1.50 6,4149,1350184122,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:35 192.168.1.50 6,4150,1350185120,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:35 192.168.1.50 6,4151,1350185123,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:35 192.168.1.50 6,4152,1350185126,-; ? shrink_node+0x155/0x570
Dec 7 10:12:35 192.168.1.50 6,4153,1350185129,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4154,1350185132,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:35 192.168.1.50 6,4155,1350185135,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:35 192.168.1.50 6,4156,1350185138,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:35 192.168.1.50 6,4157,1350185142,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:35 192.168.1.50 6,4158,1350185144,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:35 192.168.1.50 6,4159,1350185146,-; ? policy_node+0x69/0x80
Dec 7 10:12:35 192.168.1.50 6,4160,1350185148,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:35 192.168.1.50 6,4161,1350185150,-; ? do_anonymous_page+0xda/0x3a0
Dec 7 10:12:35 192.168.1.50 6,4162,1350185153,-; ? handle_pte_fault+0x20e/0x240
Dec 7 10:12:35 192.168.1.50 6,4163,1350185155,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:35 192.168.1.50 6,4164,1350185159,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:35 192.168.1.50 6,4165,1350185161,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:35 192.168.1.50 6,4166,1350185163,-; ? exit_to_user_mode_loop+0xe0/0x130
Dec 7 10:12:35 192.168.1.50 6,4167,1350185166,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:35 192.168.1.50 6,4168,1350202913,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:35 192.168.1.50 6,4169,1350203887,-; </TASK>
Dec 7 10:12:50 192.168.1.50 3,4170,1365438203,-;bcachefs (nvme1n1 inum 805548233 offset 2707456): data checksum error: expected 5097:77d52aa909e3dee5 got dd64:aa072c28992c4ec0 (type chacha20_poly1305_80)
Dec 7 10:12:59 192.168.1.50 3,4171,1374608913,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 5-.... } 6 jiffies s: 697 root: 0x20/.
Dec 7 10:12:59 192.168.1.50 3,4172,1374609919,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:59 192.168.1.50 6,4173,1374610902,-;Task dump for CPU 5:
Dec 7 10:12:59 192.168.1.50 6,4174,1374611888,-;task: kgx state:R running task stack: 0 pid: 5431 ppid: 4577 flags:0x00004006
Dec 7 10:12:59 192.168.1.50 6,4175,1374612901,-;Call Trace:
Dec 7 10:12:59 192.168.1.50 6,4176,1374613904,-; <TASK>
Dec 7 10:12:59 192.168.1.50 6,4177,1374614908,-; ? mutex_spin_on_owner+0x8f/0xf0
Dec 7 10:12:59 192.168.1.50 6,4178,1374615912,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:59 192.168.1.50 6,4179,1374616919,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:59 192.168.1.50 6,4180,1374617924,-; ? bch2_btree_key_cache_scan+0x23e/0x410 [bcachefs]
Dec 7 10:12:59 192.168.1.50 6,4181,1374618957,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:59 192.168.1.50 6,4182,1374619983,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:59 192.168.1.50 6,4183,1374620996,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:59 192.168.1.50 6,4184,1374622002,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:59 192.168.1.50 6,4185,1374623007,-; ? shrink_node+0x155/0x570
Dec 7 10:12:59 192.168.1.50 6,4186,1374624010,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:59 192.168.1.50 6,4187,1374625018,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:59 192.168.1.50 6,4188,1374626020,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:59 192.168.1.50 6,4189,1374627022,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:59 192.168.1.50 6,4190,1374628030,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:59 192.168.1.50 6,4191,1374629040,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:59 192.168.1.50 6,4192,1374630462,-; ? policy_node+0x69/0x80
Dec 7 10:12:59 192.168.1.50 6,4193,1374631465,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:59 192.168.1.50 6,4194,1374632465,-; ? wp_page_copy+0x3ce/0x720
Dec 7 10:12:59 192.168.1.50 6,4195,1374633469,-; ? do_wp_page+0xc4/0x410
Dec 7 10:12:59 192.168.1.50 6,4196,1374634465,-; ? handle_pte_fault+0x21b/0x240
Dec 7 10:12:59 192.168.1.50 6,4197,1374635462,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:59 192.168.1.50 6,4198,1374636463,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:59 192.168.1.50 6,4199,1374637470,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:59 192.168.1.50 6,4200,1374638478,-; ? __x64_sys_poll+0xc7/0x150
Dec 7 10:12:59 192.168.1.50 6,4201,1374639488,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:59 192.168.1.50 6,4202,1374640501,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:59 192.168.1.50 6,4203,1374641507,-; </TASK>
Dec 7 10:12:59 192.168.1.50 3,4204,1374664919,-;rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 2-.... } 6 jiffies s: 701 root: 0x4/.
Dec 7 10:12:59 192.168.1.50 3,4205,1374665984,-;rcu: blocking rcu_node structures (internal RCU debug):
Dec 7 10:12:59 192.168.1.50 6,4206,1374667029,-;Task dump for CPU 2:
Dec 7 10:12:59 192.168.1.50 6,4207,1374668076,-;task: grpc_global_tim state:R running task stack: 0 pid: 5482 ppid: 4577 flags:0x0000400c
Dec 7 10:12:59 192.168.1.50 6,4208,1374669150,-;Call Trace:
Dec 7 10:12:59 192.168.1.50 6,4209,1374670218,-; <TASK>
Dec 7 10:12:59 192.168.1.50 6,4210,1374671283,-; ? mutex_spin_on_owner+0x8f/0xf0
Dec 7 10:12:59 192.168.1.50 6,4211,1374672352,-; ? sysvec_apic_timer_interrupt+0xa6/0xd0
Dec 7 10:12:59 192.168.1.50 6,4212,1374673428,-; ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
Dec 7 10:12:59 192.168.1.50 6,4213,1374674504,-; ? bch2_btree_key_cache_scan+0x23e/0x410 [bcachefs]
Dec 7 10:12:59 192.168.1.50 6,4214,1374675602,-; ? bch2_btree_key_cache_scan+0x1fa/0x410 [bcachefs]
Dec 7 10:12:59 192.168.1.50 6,4215,1374676692,-; ? do_shrink_slab+0x15a/0x310
Dec 7 10:12:59 192.168.1.50 6,4216,1374677773,-; ? shrink_slab+0xaa/0x120
Dec 7 10:12:59 192.168.1.50 6,4217,1374678851,-; ? shrink_node_memcgs+0x17c/0x1d0
Dec 7 10:12:59 192.168.1.50 6,4218,1374679927,-; ? shrink_node+0x155/0x570
Dec 7 10:12:59 192.168.1.50 6,4219,1374681005,-; ? shrink_zones.constprop.0+0x90/0x2a0
Dec 7 10:12:59 192.168.1.50 6,4220,1374682068,-; ? do_try_to_free_pages+0x6b/0x320
Dec 7 10:12:59 192.168.1.50 6,4221,1374683123,-; ? try_to_free_pages+0xe3/0x210
Dec 7 10:12:59 192.168.1.50 6,4222,1374684174,-; ? __alloc_pages_slowpath.constprop.0+0x333/0xc50
Dec 7 10:12:59 192.168.1.50 6,4223,1374685226,-; ? __alloc_pages+0x311/0x340
Dec 7 10:12:59 192.168.1.50 6,4224,1374686260,-; ? __folio_alloc+0x1d/0x60
Dec 7 10:12:59 192.168.1.50 6,4225,1374687297,-; ? policy_node+0x69/0x80
Dec 7 10:12:59 192.168.1.50 6,4226,1374688331,-; ? vma_alloc_folio+0x91/0x320
Dec 7 10:12:59 192.168.1.50 6,4227,1374689369,-; ? do_anonymous_page+0xda/0x3a0
Dec 7 10:12:59 192.168.1.50 6,4228,1374690402,-; ? handle_pte_fault+0x20e/0x240
Dec 7 10:12:59 192.168.1.50 6,4229,1374691436,-; ? __handle_mm_fault+0x64a/0x710
Dec 7 10:12:59 192.168.1.50 6,4230,1374692472,-; ? handle_mm_fault+0xba/0x2a0
Dec 7 10:12:59 192.168.1.50 6,4231,1374693513,-; ? run_rebalance_domains+0x5c/0x80
Dec 7 10:12:59 192.168.1.50 6,4232,1374694550,-; ? do_user_addr_fault+0x1c1/0x6c0
Dec 7 10:12:59 192.168.1.50 6,4233,1374695586,-; ? exc_page_fault+0x80/0x1b0
Dec 7 10:12:59 192.168.1.50 6,4234,1374696619,-; ? asm_exc_page_fault+0x27/0x30
Dec 7 10:12:59 192.168.1.50 6,4235,1374697658,-; </TASK>
Dec 7 10:13:05 192.168.1.50 3,4236,1380662843,-;bcachefs (nvme1n1 inum 805548233 offset 13103104): data checksum error: expected 3f5c:663ecbdd432198fc got ef50:b635e6840ebc2d0e (type chacha20_poly1305_80)
Dec 7 10:13:56 192.168.1.50 6,4237,1431490866,-;sysrq: Resetting
