tetragon
tetragon copied to clipboard
cilium
stop test when RCU stalls happen
I was hoping that https://github.com/cilium/tetragon/commit/88128417dd1abbea900d2fed9994f906ba570ea5 would cause a panic when an RCU stall happens, which would result in the vm being stopped since we are setting panic=-1 in the VM's kernel args: https://github.com/cilium/tetragon/blob/b8ad0ac9636c9259407f738beab99f43041aefda/cmd/tetragon-vmtests-run/qemu.go#L45
to reboot immediately. see https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt:
panic= [KNL] Kernel behaviour on panic: delay
timeout > 0: seconds before rebooting timeout = 0: wait forever timeout < 0: reboot immediately Format:
However, it does not seem to work. Run https://github.com/cilium/tetragon/runs/7868377256?check_suite_focus=true, had RCU stalls but the machine did not reboot.
kernel log:
Decompressing Linux... Parsing ELF... Performing relocations... done.
Booting the kernel.
[ 0.000000] Linux version 5.4.209 (root@eecd43a3ba4d) (gcc version 12.1.0 (Debian 12.1.0-7)) #1 SMP Thu Aug 4 16:31:52 UTC 2022
[ 0.000000] Command line: root=/dev/sda console=ttyS0 earlyprintk=ttyS0 panic=-1
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdffff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000bffe0000-0x00000000bfffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable
[ 0.000000] printk: bootconsole [earlyser0] enabled
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 2095.062 MHz processor
[ 0.004524] last_pfn = 0x140000 max_arch_pfn = 0x400000000
[ 0.005869] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
Memory KASLR using RDTSC...
[ 0.006886] last_pfn = 0xbffe0 max_arch_pfn = 0x400000000
[ 0.021041] found SMP MP-table at [mem 0x000f5c80-0x000f5c8f]
[ 0.023197] check: Scanning 1 areas for low memory corruption
[ 0.028920] ACPI: Early table checksum verification disabled
[ 0.029535] ACPI: RSDP 0x00000000000F5A50 000014 (v00 BOCHS )
[ 0.029949] ACPI: RSDT 0x00000000BFFE15A2 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.030912] ACPI: FACP 0x00000000BFFE1476 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.031724] ACPI: DSDT 0x00000000BFFE0040 001436 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.032027] ACPI: FACS 0x00000000BFFE0000 000040
[ 0.032200] ACPI: APIC 0x00000000BFFE14EA 000080 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.032407] ACPI: HPET 0x00000000BFFE156A 000038 (v01 BOCHS BXPCHPET 00000001 BXPC 00000001)
[ 0.032716] ACPI: Reserving FACP table memory at [mem 0xbffe1476-0xbffe14e9]
[ 0.032938] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1475]
[ 0.033095] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]
[ 0.033248] ACPI: Reserving APIC table memory at [mem 0xbffe14ea-0xbffe1569]
[ 0.033400] ACPI: Reserving HPET table memory at [mem 0xbffe156a-0xbffe15a1]
[ 0.036371] No NUMA configuration found
[ 0.036501] Faking a node at [mem 0x0000000000000000-0x000000013fffffff]
[ 0.037510] NODE_DATA(0) allocated [mem 0x13fff9000-0x13fffcfff]
[ 0.041020] Zone ranges:
[ 0.041175] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.041403] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.041556] Normal [mem 0x0000000100000000-0x000000013fffffff]
[ 0.041721] Movable zone start for each node
[ 0.041867] Early memory node ranges
[ 0.041984] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.042258] node 0: [mem 0x0000000000100000-0x00000000bffdffff]
[ 0.042429] node 0: [mem 0x0000000100000000-0x000000013fffffff]
[ 0.044055] Zeroed struct page in unavailable ranges: 130 pages
[ 0.044265] Initmem setup node 0 [mem 0x0000000000001000-0x000000013fffffff]
[ 0.194375] ACPI: PM-Timer IO Port: 0x608
[ 0.195162] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.195786] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[ 0.196095] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.196564] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.196764] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.197024] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.197176] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.197546] Using ACPI (MADT) for SMP configuration information
[ 0.197729] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[ 0.198238] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.199398] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.199611] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.199777] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.199918] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.200064] PM: Registered nosave memory: [mem 0xbffe0000-0xbfffffff]
[ 0.200205] PM: Registered nosave memory: [mem 0xc0000000-0xfffbffff]
[ 0.200345] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
[ 0.200615] [mem 0xc0000000-0xfffbffff] available for PCI devices
[ 0.201266] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.870566] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:2 nr_node_ids:1
[ 0.881357] percpu: Embedded 56 pages/cpu s190552 r8192 d30632 u1048576
[ 0.885069] Built 1 zonelists, mobility grouping on. Total pages: 1032041
[ 0.885265] Policy zone: Normal
[ 0.885580] Kernel command line: root=/dev/sda console=ttyS0 earlyprintk=ttyS0 panic=-1
[ 0.902561] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.910582] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.912927] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 1.246979] Memory: 4026852K/4193784K available (12291K kernel code, 1312K rwdata, 6036K rodata, 1272K init, 2092K bss, 166932K reserved, 0K cma-reserved)
[ 1.253885] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 1.265343] rcu: Hierarchical RCU implementation.
[ 1.265499] rcu: RCU event tracing is enabled.
[ 1.265641] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 1.266018] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[ 1.266227] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 1.292085] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 1.303950] Console: colour *CGA 80x25
[ 1.305876] printk: console [ttyS0] enabled
[ 1.305876] printk: console [ttyS0] enabled
[ 1.331125] printk: bootconsole [earlyser0] disabled
[ 1.331125] printk: bootconsole [earlyser0] disabled
[ 1.332384] ACPI: Core revision 20190816
[ 1.338428] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[ 1.344281] APIC: Switch to symmetric I/O mode setup
[ 1.349496] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 1.355190] tsc: Marking TSC unstable due to TSCs unsynchronized
[ 1.355728] Calibrating delay loop (skipped), value calculated using timer frequency.. 4190.12 BogoMIPS (lpj=2095062)
[ 1.356894] pid_max: default: 32768 minimum: 301
[ 1.357962] LSM: Security Framework initializing
[ 1.359068] SELinux: Initializing.
[ 1.360197] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 1.360491] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
Poking KASLR using RDTSC...
[ 1.384884] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 1.385071] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 1.385527] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 1.385826] Spectre V2 : Mitigation: Retpolines
[ 1.385950] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 1.386202] Speculative Store Bypass: Vulnerable
[ 1.393236] Freeing SMP alternatives memory: 36K
[ 1.511458] smpboot: CPU0: AMD QEMU Virtual CPU version 2.5+ (family: 0x6, model: 0x6, stepping: 0x3)
[ 1.519234] Performance Events: PMU not available due to virtualization, using software events only.
[ 1.521656] rcu: Hierarchical SRCU implementation.
[ 1.525060] Huh? What family is it: 0x6?!
[ 1.527094] smp: Bringing up secondary CPUs ...
[ 1.530647] x86: Booting SMP configuration:
[ 1.530831] .... node #0, CPUs: #1
[ 1.607651] smp: Brought up 1 node, 2 CPUs
[ 1.607955] smpboot: Max logical packages: 2
[ 1.608132] smpboot: Total of 2 processors activated (8380.92 BogoMIPS)
[ 1.626557] devtmpfs: initialized
[ 1.637003] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 1.637438] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 1.641458] PM: RTC time: 22:46:03, date: 2022-08-16
[ 1.648145] NET: Registered protocol family 16
[ 1.651120] audit: initializing netlink subsys (disabled)
[ 1.653058] audit: type=2000 audit(1660689963.309:1): state=initialized audit_enabled=0 res=1
[ 1.660128] cpuidle: using governor menu
[ 1.661768] ACPI: bus type PCI registered
[ 1.664306] PCI: Using configuration type 1 for base access
[ 1.667834] mtrr: your CPUs had inconsistent fixed MTRR settings
[ 1.668253] mtrr: your CPUs had inconsistent variable MTRR settings
[ 1.668513] mtrr: your CPUs had inconsistent MTRRdefType settings
[ 1.668782] mtrr: probably your BIOS does not setup all CPUs.
[ 1.669015] mtrr: corrected configuration.
[ 1.688118] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 1.693411] cryptomgr_test (26) used greatest stack depth: 15520 bytes left
[ 1.694755] cryptomgr_test (28) used greatest stack depth: 15216 bytes left
[ 1.706128] ACPI: Added _OSI(Module Device)
[ 1.706518] ACPI: Added _OSI(Processor Device)
[ 1.706641] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 1.706776] ACPI: Added _OSI(Processor Aggregator Device)
[ 1.707114] ACPI: Added _OSI(Linux-Dell-Video)
[ 1.707233] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[ 1.707480] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[ 1.724336] ACPI: 1 ACPI AML tables successfully acquired and loaded
[ 1.742197] ACPI: Interpreter enabled
[ 1.743305] ACPI: (supports S0 S3 S4 S5)
[ 1.743495] ACPI: Using IOAPIC for interrupt routing
[ 1.744110] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 1.746117] ACPI: Enabled 2 GPEs in block 00 to 0F
[ 1.785458] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 1.786271] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3]
[ 1.787565] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 1.790349] PCI host bridge to bus 0000:00
[ 1.790658] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 1.790870] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 1.791081] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 1.791251] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[ 1.791428] pci_bus 0000:00: root bus resource [mem 0x140000000-0x1bfffffff window]
[ 1.791458] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.792028] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[ 1.797398] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[ 1.798726] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[ 1.803096] pci 0000:00:01.1: reg 0x20: [io 0xc080-0xc08f]
[ 1.804969] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 1.805544] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 1.805963] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 1.806505] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 1.808007] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[ 1.809206] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 1.809545] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 1.811906] pci 0000:00:02.0: [1af4:1003] type 00 class 0x078000
[ 1.813039] pci 0000:00:02.0: reg 0x10: [io 0xc000-0xc03f]
[ 1.814519] pci 0000:00:02.0: reg 0x14: [mem 0xfebfe000-0xfebfefff]
[ 1.820458] pci 0000:00:02.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref]
[ 1.823667] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200
[ 1.825113] pci 0000:00:03.0: reg 0x10: [io 0xc040-0xc07f]
[ 1.827114] pci 0000:00:03.0: reg 0x14: [mem 0xfebff000-0xfebfffff]
[ 1.832512] pci 0000:00:03.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref]
[ 1.841799] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[ 1.842889] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[ 1.843664] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[ 1.844363] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[ 1.844458] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
[ 1.847623] iommu: Default domain type: Translated
[ 1.852238] vgaarb: loaded
[ 1.854492] SCSI subsystem initialized
[ 1.857110] pps_core: LinuxPPS API ver. 1 registered
[ 1.857277] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[ 1.857504] PTP clock support registered
[ 1.858707] EDAC MC: Ver: 3.0.0
[ 1.862008] PCI: Using ACPI for IRQ routing
[ 1.871320] NetLabel: Initializing
[ 1.871491] NetLabel: domain hash size = 128
[ 1.871615] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 1.873394] NetLabel: unlabeled traffic allowed by default
[ 1.873892] hpet: 3 channels of 0 reserved for per-cpu timers
[ 1.874531] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 1.874732] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[ 1.879053] clocksource: Switched to clocksource hpet
[ 2.362741] VFS: Disk quotas dquot_6.6.0
[ 2.363155] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 2.366396] pnp: PnP ACPI init
[ 2.372577] pnp: PnP ACPI: found 5 devices
[ 2.399005] thermal_sys: Registered thermal governor 'step_wise'
[ 2.399124] thermal_sys: Registered thermal governor 'user_space'
[ 2.404190] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 2.405451] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 2.405640] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 2.405793] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[ 2.405925] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[ 2.405925] pci_bus 0000:00: resource 8 [mem 0x140000000-0x1bfffffff window]
[ 2.407555] NET: Registered protocol family 2
[ 2.409401] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 2.416761] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 2.417266] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 2.418156] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[ 2.420332] TCP: Hash tables configured (established 32768 bind 32768)
[ 2.421963] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 2.422467] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 2.424405] NET: Registered protocol family 1
[ 2.428307] RPC: Registered named UNIX socket transport module.
[ 2.428498] RPC: Registered udp transport module.
[ 2.428498] RPC: Registered tcp transport module.
[ 2.428498] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 2.428498] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 2.428498] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 2.428498] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 2.428498] PCI: CLS 0 bytes, default 64
[ 2.435083] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 2.435344] software IO TLB: mapped [mem 0xbbfe0000-0xbffe0000] (64MB)
[ 2.442439] check: Scanning for low memory corruption every 60 seconds
[ 2.446989] Initialise system trusted keyrings
[ 2.449113] workingset: timestamp_bits=56 max_order=20 bucket_order=0
[ 2.466969] NFS: Registering the id_resolver key type
[ 2.467488] Key type id_resolver registered
[ 2.467507] Key type id_legacy registered
[ 2.468727] 9p: Installing v9fs 9p2000 file system support
[ 2.489442] Key type asymmetric registered
[ 2.489756] Asymmetric key parser 'x509' registered
[ 2.490129] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 2.490449] io scheduler mq-deadline registered
[ 2.490629] io scheduler kyber registered
[ 2.496009] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 2.500183] ACPI: Power Button [PWRF]
[ 2.681640] PCI Interrupt Link [LNKB] enabled at IRQ 10
[ 2.858684] PCI Interrupt Link [LNKC] enabled at IRQ 11
[ 2.861703] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 2.885329] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 2.891386] Non-volatile memory driver v1.3
[ 2.919654] loop: module loaded
[ 2.930417] scsi host0: ata_piix
[ 2.933292] scsi host1: ata_piix
[ 2.934384] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc080 irq 14
[ 2.934634] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc088 irq 15
[ 2.936280] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
[ 2.939430] e100: Copyright(c) 1999-2006 Intel Corporation
[ 2.940372] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[ 2.940372] e1000: Copyright (c) 1999-2006 Intel Corporation.
[ 2.940372] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[ 2.940372] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 2.940372] sky2: driver version 1.30
[ 2.947367] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 2.951798] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 2.952383] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 2.956977] rtc_cmos 00:00: RTC can wake from S4
[ 2.960877] rtc_cmos 00:00: registered as rtc0
[ 2.961915] rtc_cmos 00:00: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 2.963600] device-mapper: ioctl: 4.41.0-ioctl (2019-09-16) initialised: [email protected]
[ 2.974671] Initializing XFRM netlink socket
[ 2.976387] NET: Registered protocol family 10
[ 2.984968] Segment Routing with IPv6
[ 2.988651] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 2.992276] NET: Registered protocol family 17
[ 2.994448] 9pnet: Installing 9P2000 support
[ 3.001046] Key type dns_resolver registered
[ 3.003336] IPI shorthand broadcast: enabled
[ 3.004810] registered taskstats version 1
[ 3.004958] Loading compiled-in X.509 certificates
[ 3.010127] PM: Magic number: 2:785:805
[ 3.010896] printk: console [netcon0] enabled
[ 3.011094] netconsole: network logging started
[ 3.014331] Unstable clock detected, switching default tracing clock to "global"
[ 3.014331] If you want to keep using the local clock, then add:
[ 3.014331] "trace_clock=local"
[ 3.014331] on the kernel command line
[ 4.409171] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3
[ 22.853587] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 22.854256] rcu: 1-...!: (0 ticks this GP) idle=ca4/0/0x0 softirq=20/20 fqs=0
[ 22.854496] (detected by 0, t=21002 jiffies, g=-1139, q=39)
[ 22.854496] Sending NMI from CPU 0 to CPUs 1:
[ 22.859421] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 22.859546] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 2.120 msecs
[ 22.854496] rcu: rcu_sched kthread starved for 21002 jiffies! g-1139 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 22.854496] rcu: RCU grace-period kthread stack dump:
[ 22.854496] rcu_sched I15016 11 2 0x80004000
[ 22.854496] Call Trace:
[ 22.854496] __schedule+0x288/0x600
[ 22.854496] ? __mod_timer+0x1a6/0x3c0
[ 22.854496] schedule+0x34/0xa0
[ 22.854496] schedule_timeout+0x84/0x140
[ 22.854496] ? __next_timer_interrupt+0xc0/0xc0
[ 22.854496] rcu_gp_kthread+0x4f6/0xd40
[ 22.854496] ? kfree_call_rcu+0x10/0x10
[ 22.854496] kthread+0x107/0x120
[ 22.854496] ? __kthread_bind_mask+0x60/0x60
[ 22.854496] ret_from_fork+0x35/0x40
[ 43.855505] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 43.855976] rcu: 1-...!: (0 ticks this GP) idle=cc0/0/0x0 softirq=20/20 fqs=0
[ 43.856153] (detected by 0, t=21002 jiffies, g=-1135, q=37)
[ 43.856314] Sending NMI from CPU 0 to CPUs 1:
[ 43.856538] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 43.856486] rcu: rcu_sched kthread starved for 21002 jiffies! g-1135 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 43.856486] rcu: RCU grace-period kthread stack dump:
[ 43.856486] rcu_sched I15016 11 2 0x80004000
[ 43.856486] Call Trace:
[ 43.856486] __schedule+0x288/0x600
[ 43.856486] ? __mod_timer+0x1a6/0x3c0
[ 43.856486] schedule+0x34/0xa0
[ 43.856486] schedule_timeout+0x84/0x140
[ 43.856486] ? __next_timer_interrupt+0xc0/0xc0
[ 43.856486] rcu_gp_kthread+0x4f6/0xd40
[ 43.856486] ? kfree_call_rcu+0x10/0x10
[ 43.856486] kthread+0x107/0x120
[ 43.856486] ? __kthread_bind_mask+0x60/0x60
[ 43.856486] ret_from_fork+0x35/0x40
[ 64.857512] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 64.857740] rcu: 1-...!: (0 ticks this GP) idle=cdc/0/0x0 softirq=20/20 fqs=0
[ 64.857913] (detected by 0, t=21002 jiffies, g=-1131, q=37)
[ 64.858070] Sending NMI from CPU 0 to CPUs 1:
[ 64.858282] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 64.858488] rcu: rcu_sched kthread starved for 21002 jiffies! g-1131 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 64.858488] rcu: RCU grace-period kthread stack dump:
[ 64.858488] rcu_sched I15016 11 2 0x80004000
[ 64.858488] Call Trace:
[ 64.858488] __schedule+0x288/0x600
[ 64.858488] ? __mod_timer+0x1a6/0x3c0
[ 64.858488] schedule+0x34/0xa0
[ 64.858488] schedule_timeout+0x84/0x140
[ 64.858488] ? __next_timer_interrupt+0xc0/0xc0
[ 64.858488] rcu_gp_kthread+0x4f6/0xd40
[ 64.858488] ? kfree_call_rcu+0x10/0x10
[ 64.858488] kthread+0x107/0x120
[ 64.858488] ? __kthread_bind_mask+0x60/0x60
[ 64.858488] ret_from_fork+0x35/0x40
[ 103.087243] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 103.087558] rcu: 1-...!: (0 ticks this GP) idle=cf8/0/0x0 softirq=20/20 fqs=0
[ 103.087821] (detected by 0, t=38227 jiffies, g=-1127, q=10)
[ 103.088024] Sending NMI from CPU 0 to CPUs 1:
[ 103.088684] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 103.088153] rcu: rcu_sched kthread starved for 38227 jiffies! g-1127 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 103.088153] rcu: RCU grace-period kthread stack dump:
[ 103.088153] rcu_sched I15016 11 2 0x80004000
[ 103.088153] Call Trace:
[ 103.088153] __schedule+0x288/0x600
[ 103.088153] ? __mod_timer+0x1a6/0x3c0
[ 103.088153] schedule+0x34/0xa0
[ 103.088153] schedule_timeout+0x84/0x140
[ 103.088153] ? __next_timer_interrupt+0xc0/0xc0
[ 103.088153] rcu_gp_kthread+0x4f6/0xd40
[ 103.088153] ? kfree_call_rcu+0x10/0x10
[ 103.088153] kthread+0x107/0x120
[ 103.088153] ? __kthread_bind_mask+0x60/0x60
[ 103.088153] ret_from_fork+0x35/0x40
[ 182.556239] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 182.556604] rcu: 1-...!: (0 ticks this GP) idle=d0c/0/0x0 softirq=20/20 fqs=1
[ 182.556906] (detected by 0, t=117696 jiffies, g=-1127, q=10)
[ 182.557153] Sending NMI from CPU 0 to CPUs 1:
[ 182.559019] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 182.557153] rcu: rcu_sched kthread starved for 79461 jiffies! g-1127 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 182.557153] rcu: RCU grace-period kthread stack dump:
[ 182.557153] rcu_sched I15016 11 2 0x80004000
[ 182.557153] Call Trace:
[ 182.557153] __schedule+0x288/0x600
[ 182.557153] ? __mod_timer+0x1a6/0x3c0
[ 182.557153] schedule+0x34/0xa0
[ 182.557153] schedule_timeout+0x84/0x140
[ 182.557153] ? __next_timer_interrupt+0xc0/0xc0
[ 182.557153] rcu_gp_kthread+0x4f6/0xd40
[ 182.557153] ? kfree_call_rcu+0x10/0x10
[ 182.557153] kthread+0x107/0x120
[ 182.557153] ? __kthread_bind_mask+0x60/0x60
[ 182.557153] ret_from_fork+0x35/0x40
[ 334.112602] kworker/dying (5) used greatest stack depth: 15208 bytes left
[ 355.114517] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 355.114827] rcu: 1-...!: (0 ticks this GP) idle=d28/0/0x0 softirq=20/20 fqs=0
[ 355.114999] (detected by 0, t=21002 jiffies, g=-1123, q=12)
[ 355.115193] Sending NMI from CPU 0 to CPUs 1:
[ 355.115430] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 355.115489] rcu: rcu_sched kthread starved for 21002 jiffies! g-1123 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 355.115489] rcu: RCU grace-period kthread stack dump:
[ 355.115489] rcu_sched I15016 11 2 0x80004000
[ 355.115489] Call Trace:
[ 355.115489] __schedule+0x288/0x600
[ 355.115489] ? __mod_timer+0x1a6/0x3c0
[ 355.115489] schedule+0x34/0xa0
[ 355.115489] schedule_timeout+0x84/0x140
[ 355.115489] ? __next_timer_interrupt+0xc0/0xc0
[ 355.115489] rcu_gp_kthread+0x4f6/0xd40
[ 355.115489] ? kfree_call_rcu+0x10/0x10
[ 355.115489] kthread+0x107/0x120
[ 355.115489] ? __kthread_bind_mask+0x60/0x60
[ 355.115489] ret_from_fork+0x35/0x40
[ 376.116517] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 376.117044] rcu: 1-...!: (0 ticks this GP) idle=d48/0/0x0 softirq=20/20 fqs=0
[ 376.117250] (detected by 0, t=21002 jiffies, g=-1119, q=11)
[ 376.117432] Sending NMI from CPU 0 to CPUs 1:
[ 376.117645] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 376.117497] rcu: rcu_sched kthread starved for 21002 jiffies! g-1119 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 376.117497] rcu: RCU grace-period kthread stack dump:
[ 376.117497] rcu_sched I15016 11 2 0x80004000
[ 376.117497] Call Trace:
[ 376.117497] __schedule+0x288/0x600
[ 376.117497] ? __mod_timer+0x1a6/0x3c0
[ 376.117497] schedule+0x34/0xa0
[ 376.117497] schedule_timeout+0x84/0x140
[ 376.117497] ? __next_timer_interrupt+0xc0/0xc0
[ 376.117497] rcu_gp_kthread+0x4f6/0xd40
[ 376.117497] ? kfree_call_rcu+0x10/0x10
[ 376.117497] kthread+0x107/0x120
[ 376.117497] ? __kthread_bind_mask+0x60/0x60
[ 376.117497] ret_from_fork+0x35/0x40
[ 397.118507] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 397.118932] rcu: 1-...!: (0 ticks this GP) idle=d64/0/0x0 softirq=20/20 fqs=0
[ 397.119124] (detected by 0, t=21002 jiffies, g=-1115, q=11)
[ 397.119285] Sending NMI from CPU 0 to CPUs 1:
[ 397.119499] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 397.119488] rcu: rcu_sched kthread starved for 21002 jiffies! g-1115 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 397.119488] rcu: RCU grace-period kthread stack dump:
[ 397.119488] rcu_sched I15016 11 2 0x80004000
[ 397.119488] Call Trace:
[ 397.119488] __schedule+0x288/0x600
[ 397.119488] ? __mod_timer+0x1a6/0x3c0
[ 397.119488] schedule+0x34/0xa0
[ 397.119488] schedule_timeout+0x84/0x140
[ 397.119488] ? __next_timer_interrupt+0xc0/0xc0
[ 397.119488] rcu_gp_kthread+0x4f6/0xd40
[ 397.119488] ? kfree_call_rcu+0x10/0x10
[ 397.119488] kthread+0x107/0x120
[ 397.119488] ? __kthread_bind_mask+0x60/0x60
[ 397.119488] ret_from_fork+0x35/0x40
[ 418.120507] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 418.120740] rcu: 1-...!: (0 ticks this GP) idle=d80/0/0x0 softirq=20/20 fqs=0
[ 418.120912] (detected by 0, t=21002 jiffies, g=-1111, q=11)
[ 418.121066] Sending NMI from CPU 0 to CPUs 1:
[ 418.121399] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 418.121486] rcu: rcu_sched kthread starved for 21002 jiffies! g-1111 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 418.121486] rcu: RCU grace-period kthread stack dump:
[ 418.121486] rcu_sched I15016 11 2 0x80004000
[ 418.121486] Call Trace:
[ 418.121486] __schedule+0x288/0x600
[ 418.121486] ? __mod_timer+0x1a6/0x3c0
[ 418.121486] schedule+0x34/0xa0
[ 418.121486] schedule_timeout+0x84/0x140
[ 418.121486] ? __next_timer_interrupt+0xc0/0xc0
[ 418.121486] rcu_gp_kthread+0x4f6/0xd40
[ 418.121486] ? kfree_call_rcu+0x10/0x10
[ 418.121486] kthread+0x107/0x120
[ 418.121486] ? __kthread_bind_mask+0x60/0x60
[ 418.121486] ret_from_fork+0x35/0x40
[ 451.528232] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 451.528613] rcu: 1-...!: (0 ticks this GP) idle=d9c/0/0x0 softirq=20/20 fqs=0
[ 451.528922] (detected by 0, t=33407 jiffies, g=-1107, q=10)
[ 451.529147] Sending NMI from CPU 0 to CPUs 1:
[ 451.532987] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 451.529147] rcu: rcu_sched kthread starved for 33407 jiffies! g-1107 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 451.529147] rcu: RCU grace-period kthread stack dump:
[ 451.529147] rcu_sched I15016 11 2 0x80004000
[ 451.529147] Call Trace:
[ 451.529147] __schedule+0x288/0x600
[ 451.529147] ? __mod_timer+0x1a6/0x3c0
[ 451.529147] schedule+0x34/0xa0
[ 451.529147] schedule_timeout+0x84/0x140
[ 451.529147] ? __next_timer_interrupt+0xc0/0xc0
[ 451.529147] rcu_gp_kthread+0x4f6/0xd40
[ 451.529147] ? kfree_call_rcu+0x10/0x10
[ 451.529147] kthread+0x107/0x120
[ 451.529147] ? __kthread_bind_mask+0x60/0x60
[ 451.529147] ret_from_fork+0x35/0x40
[ 532.081249] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 532.081510] rcu: 1-...!: (0 ticks this GP) idle=db0/0/0x0 softirq=20/20 fqs=1
[ 532.081735] (detected by 0, t=113960 jiffies, g=-1107, q=10)
[ 532.081927] Sending NMI from CPU 0 to CPUs 1:
[ 532.082983] NMI backtrace for cpu 1 skipped: idling at default_idle+0x1e/0x130
[ 532.082138] rcu: rcu_sched kthread starved for 80545 jiffies! g-1107 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 532.082138] rcu: RCU grace-period kthread stack dump:
[ 532.082138] rcu_sched I15016 11 2 0x80004000
[ 532.082138] Call Trace:
[ 532.082138] __schedule+0x288/0x600
[ 532.082138] ? __mod_timer+0x1a6/0x3c0
[ 532.082138] schedule+0x34/0xa0
[ 532.082138] schedule_timeout+0x84/0x140
[ 532.082138] ? __next_timer_interrupt+0xc0/0xc0
[ 532.082138] rcu_gp_kthread+0x4f6/0xd40
[ 532.082138] ? kfree_call_rcu+0x10/0x10
[ 532.082138] kthread+0x107/0x120
[ 532.082138] ? __kthread_bind_mask+0x60/0x60
[ 532.082138] ret_from_fork+0x35/0x40
