mce-inject
mce-inject copied to clipboard
andikleen
Why the output of "mce-inject uncorrect_error" is different when boot with "quiet" and without "quiet"
# cat uncorrect_error
# this should just kill the process
CPU 0 BANK 1
STATUS UNCORRECTED
ADDR 0x1234
When boot without "quiet" and then run "mce-inject uncorrect_error", the output is expected as below:
# mce-inject uncorrect_error
[ 270.522201] mce: Triggering MCE exception on CPU 0
[ 270.522300] Disabling lock debugging due to kernel taint
[ 270.522390] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 1: b400000000000000
[ 273.358647] mce: [Hardware Error]: TSC 1d6086030ad ADDR 1234
[ 273.365062] mce: [Hardware Error]: PROCESSOR 0:406f0 TIME 1736134149 SOCKET 0 APIC 0 microcode 10
[ 273.374967] mce: [Hardware Error]: Run the above through 'mcelog --ascii'
[ 273.382543] mce: [Hardware Error]: Machine check: MCIP not set in MCA handler
[ 273.390507] Kernel panic - not syncing:
[ 273.394787] Fatal machine check
[ 273.394790] CPU: 44 PID: 0 Comm: swapper/44 Tainted: G M 5.15.169-rt76-yocto-preempt-rt #1
[ 273.408976] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRRFSDP1.86B.0251.R01.1505050612 05/05/2015
[ 273.420336] Call Trace:
[ 273.423066] <NMI>
[ 273.425312] dump_stack_lvl+0x38/0x4d
[ 273.429407] dump_stack+0x10/0x16
[ 273.433106] panic+0xaa/0x2ed
[ 273.436423] mce_panic+0x175/0x340
[ 273.440224] mce_end+0x20c/0x290
[ 273.443828] do_machine_check+0x601/0x990
[ 273.448305] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.454436] raise_exception+0x39/0xa0
[ 273.458624] ? debug_smp_processor_id+0x17/0x20
[ 273.463685] ? intel_pmu_lbr_enable_all+0x1f/0x1d0
[ 273.469036] ? debug_smp_processor_id+0x17/0x20
[ 273.474088] ? intel_bts_enable_local+0x16/0x60
[ 273.479145] ? x2apic_cluster_probe+0x60/0x60
[ 273.484012] ? intel_pmu_handle_irq+0x25d/0x5a0
[ 273.489070] mce_raise_notify+0xc8/0xe0
[ 273.493354] nmi_handle+0x6f/0x170
[ 273.497153] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.503279] default_do_nmi+0x71/0x200
[ 273.507456] exc_nmi+0xfb/0x130
[ 273.510962] end_repeat_nmi+0x16/0x67
[ 273.515053] RIP: 0010:mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.521857] Code: e2 08 75 26 48 8b 00 a9 00 00 08 00 75 1c 8b 05 14 68 74 01 85 c0 7e 07 0f 00 2d 59 cf d4 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 80 ad 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b
[ 273.542816] RSP: 0018:ffffb34ac659be50 EFLAGS: 00000042
[ 273.548648] RAX: 0000000000000020 RBX: ffffd342bf785500 RCX: 0000000000000001
[ 273.556615] RDX: 0000000000000000 RSI: ffffffffaf3bb220 RDI: 0000000000000020
[ 273.564581] RBP: ffffb34ac659be60 R08: 0000000000030807 R09: 0000000000000018
[ 273.572545] R10: 000000000001ef15 R11: ffff894e5f7a9064 R12: 0000000000000004
[ 273.580509] R13: ffffffffaf3bb220 R14: ffffffffaf3bb3d8 R15: 0000000000000004
[ 273.588478] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.594603] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.600719] </NMI>
[ 273.603059] <TASK>
[ 273.605400] ? intel_idle+0x1f/0x30
[ 273.609294] cpuidle_enter_state+0x92/0x3b0
[ 273.613965] cpuidle_enter+0x2e/0x50
[ 273.617954] do_idle+0x22a/0x2b0
[ 273.621564] cpu_startup_entry+0x20/0x30
[ 273.625943] start_secondary+0xf3/0x100
[ 273.630229] secondary_startup_64_no_verify+0xc2/0xcb
[ 273.635871] </TASK>
[ 274.638429] Kernel Offset: 0x2c800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 274.650465] Rebooting in 30 seconds..
But when boot with "quiet" and then run "mce-inject uncorrect_error", the output includes
some message from dmesg apart from the above call trace as below.
Why the message from dmesg printed? Any hints? Many thanks!
# mce-inject uncorrect_error
[ 10.330465] systemd[1]: Queued start job for default target Multi-User System.
[ 10.422258] systemd[1]: Created slice Slice /system/getty.
[ 10.423545] systemd[1]: Created slice Slice /system/modprobe.
[ 10.424725] systemd[1]: Created slice Slice /system/serial-getty.
[ 10.426091] systemd[1]: Created slice User and Session Slice.
[ 10.426164] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[ 10.426215] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[ 10.426242] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426262] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426279] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426288] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426298] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426306] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426318] systemd[1]: Reached target ceph target allowing to start/stop all [email protected] instances at once.
[ 10.426358] systemd[1]: Reached target Path Units.
[ 10.426379] systemd[1]: Reached target Slice Units.
[ 10.426395] systemd[1]: Reached target Swaps.
[ 10.426416] systemd[1]: Reached target Libvirt guests shutdown.
[ 10.426946] systemd[1]: Listening on RPCbind Server Activation Socket.
[ 10.426965] systemd[1]: Reached target RPC Port Mapper.
[ 10.427462] systemd[1]: Listening on Syslog Socket.
[ 10.427532] systemd[1]: Listening on initctl Compatibility Named Pipe.
[ 10.443032] systemd[1]: Journal Audit Socket was skipped because of a failed condition check (ConditionSecurity=audit).
[ 10.443348] systemd[1]: Listening on Journal Socket (/dev/log).
[ 10.443487] systemd[1]: Listening on Journal Socket.
[ 10.443831] systemd[1]: Listening on Network Service Netlink Socket.
[ 10.444703] systemd[1]: Listening on udev Control Socket.
[ 10.444807] systemd[1]: Listening on udev Kernel Socket.
[ 10.445169] systemd[1]: Listening on User Database Manager Socket.
[ 10.446833] systemd[1]: Mounting Huge Pages File System...
[ 10.448587] systemd[1]: Mounting POSIX Message Queue File System...
[ 10.450292] systemd[1]: Mounting Kernel Debug File System...
[ 10.452040] systemd[1]: Mounting Kernel Trace File System...
[ 10.454795] systemd[1]: Mounting Temporary Directory /tmp...
[ 10.457264] systemd[1]: Starting Create List of Static Device Nodes...
[ 10.458717] systemd[1]: Starting Load Kernel Module configfs...
[ 10.460097] systemd[1]: Starting Load Kernel Module drm...
[ 10.461389] systemd[1]: Starting Load Kernel Module fuse...
[ 10.464251] systemd[1]: Starting RPC Bind...
[ 10.464551] systemd[1]: File System Check on Root Device was skipped because of a failed condition check (ConditionPathIsReadWrite=!/).
[ 10.464779] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.
[ 10.464782] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)
[ 10.466838] systemd[1]: Starting Journal Service...
[ 10.472028] fuse: init (API version 7.34)
[ 10.472061] systemd[1]: Starting Load Kernel Modules...
[ 10.473855] systemd[1]: Starting Remount Root and Kernel File Systems...
[ 10.475576] systemd[1]: Starting Coldplug All udev Devices...
[ 10.477211] systemd[1]: Mounted Huge Pages File System.
[ 10.477313] systemd[1]: Mounted POSIX Message Queue File System.
[ 10.477408] systemd[1]: Mounted Kernel Debug File System.
[ 10.477500] systemd[1]: Mounted Kernel Trace File System.
[ 10.477645] systemd[1]: Mounted Temporary Directory /tmp.
[ 10.478003] systemd[1]: Finished Create List of Static Device Nodes.
[ 10.478246] systemd[1]: [email protected]: Deactivated successfully.
[ 10.478470] systemd[1]: Finished Load Kernel Module configfs.
[ 10.478725] systemd[1]: [email protected]: Deactivated successfully.
[ 10.478940] systemd[1]: Finished Load Kernel Module drm.
[ 10.479189] systemd[1]: [email protected]: Deactivated successfully.
[ 10.479402] systemd[1]: Finished Load Kernel Module fuse.
[ 10.481230] systemd[1]: Mounting FUSE Control File System...
[ 10.482968] systemd[1]: Mounting Kernel Configuration File System...
[ 10.488783] systemd[1]: Mounted FUSE Control File System.
[ 10.488885] systemd[1]: Mounted Kernel Configuration File System.
[ 10.498347] systemd[1]: Started RPC Bind.
[ 10.500549] systemd[1]: Finished Remount Root and Kernel File Systems.
[ 10.513602] systemd[1]: Rebuild Hardware Database was skipped because of a failed condition check (ConditionNeedsUpdate=/etc).
[ 10.513652] systemd[1]: Platform Persistent Storage Archival was skipped because of a failed condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 10.514955] systemd[1]: Create System Users was skipped because of a failed condition check (ConditionNeedsUpdate=/etc).
[ 10.516543] systemd[1]: Starting Create Static Device Nodes in /dev...
[ 10.561764] systemd[1]: Finished Create Static Device Nodes in /dev.
[ 10.561863] systemd[1]: Reached target Preparation for Local File Systems.
[ 10.564135] systemd[1]: Starting Rule-based Manager for Device Events and Files...
[ 10.566052] openvswitch: Open vSwitch switching datapath
[ 10.567827] systemd[1]: Finished Load Kernel Modules.
[ 10.569767] systemd[1]: Starting Apply Kernel Variables...
[ 10.592423] systemd[1]: Finished Apply Kernel Variables.
[ 10.615731] systemd[1]: Started Journal Service.
[ 10.626407] systemd-journald[1133]: Received client request to flush runtime journal.
[ 10.851505] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt
[ 10.852891] i2c i2c-0: 8/24 memory slots populated (from DMI)
[ 10.852894] i2c i2c-0: Systems with more than 4 memory slots not supported yet, not instantiating SPD
[ 10.853230] mgag200 0000:07:00.0: vgaarb: deactivate vga console
[ 10.861271] Console: switching to colour dummy device 80x25
[ 10.870269] [drm] Initialized mgag200 1.0.0 20110418 for 0000:07:00.0 on minor 0
[ 10.895030] power_meter ACPI000D:00: Found ACPI power meter.
[ 10.895063] power_meter ACPI000D:00: Ignoring unsafe software power cap!
[ 10.895071] power_meter ACPI000D:00: hwmon_device_register() is deprecated. Please convert the driver to use hwmon_device_register_with_info().
[ 10.900599] cryptd: max_cpu_qlen set to 1000
[ 10.908371] fbcon: mgag200drmfb (fb0) is primary device
[ 10.911794] mgag200 0000:07:00.0: [drm] drm_plane_enable_fb_damage_clips() not called
[ 10.948976] AVX2 version of gcm_enc/dec engaged.
[ 10.949116] AES CTR mode by8 optimization enabled
[ 11.019020] Console: switching to colour frame buffer device 160x64
[ 11.056429] mgag200 0000:07:00.0: [drm] fb0: mgag200drmfb frame buffer device
[ 11.187508] iTCO_vendor_support: vendor-support=0
[ 11.317199] iTCO_wdt iTCO_wdt.1.auto: unable to reset NO_REBOOT flag, device disabled by hardware/BIOS
[ 11.502050] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
[ 11.806713] EDAC DEBUG: sbridge_init:
[ 11.806716] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
[ 11.806734] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6fa0
[ 11.806735] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
[ 11.806754] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6fa0
[ 11.806755] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
[ 11.806770] EDAC sbridge: Seeking for: PCI ID 8086:6f60
[ 11.806776] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f60
[ 11.806777] EDAC sbridge: Seeking for: PCI ID 8086:6f60
[ 11.806785] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f60
[ 11.806786] EDAC sbridge: Seeking for: PCI ID 8086:6f60
[ 11.806791] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
[ 11.806797] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6fa8
[ 11.806798] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
[ 11.806806] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6fa8
[ 11.806807] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
[ 11.806812] EDAC sbridge: Seeking for: PCI ID 8086:6f71
[ 11.806818] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f71
[ 11.806819] EDAC sbridge: Seeking for: PCI ID 8086:6f71
[ 11.806827] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f71
[ 11.806828] EDAC sbridge: Seeking for: PCI ID 8086:6f71
[ 11.806833] EDAC sbridge: Seeking for: PCI ID 8086:6faa
[ 11.806839] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6faa
[ 11.806840] EDAC sbridge: Seeking for: PCI ID 8086:6faa
[ 11.806847] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6faa
[ 11.806848] EDAC sbridge: Seeking for: PCI ID 8086:6faa
[ 11.806853] EDAC sbridge: Seeking for: PCI ID 8086:6fab
[ 11.806859] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6fab
[ 11.806860] EDAC sbridge: Seeking for: PCI ID 8086:6fab
[ 11.806868] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6fab
[ 11.806869] EDAC sbridge: Seeking for: PCI ID 8086:6fab
[ 11.806873] EDAC sbridge: Seeking for: PCI ID 8086:6fac
[ 11.806887] EDAC sbridge: Seeking for: PCI ID 8086:6fad
[ 11.806900] EDAC sbridge: Seeking for: PCI ID 8086:6f68
[ 11.806907] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f68
[ 11.806908] EDAC sbridge: Seeking for: PCI ID 8086:6f68
[ 11.806916] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f68
[ 11.806917] EDAC sbridge: Seeking for: PCI ID 8086:6f68
[ 11.806921] EDAC sbridge: Seeking for: PCI ID 8086:6f79
[ 11.806927] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f79
[ 11.806928] EDAC sbridge: Seeking for: PCI ID 8086:6f79
[ 11.806936] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f79
[ 11.806937] EDAC sbridge: Seeking for: PCI ID 8086:6f79
[ 11.806941] EDAC sbridge: Seeking for: PCI ID 8086:6f6a
[ 11.806948] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f6a
[ 11.806949] EDAC sbridge: Seeking for: PCI ID 8086:6f6a
[ 11.806957] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f6a
[ 11.806958] EDAC sbridge: Seeking for: PCI ID 8086:6f6a
[ 11.806962] EDAC sbridge: Seeking for: PCI ID 8086:6f6b
[ 11.806969] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f6b
[ 11.806970] EDAC sbridge: Seeking for: PCI ID 8086:6f6b
[ 11.806981] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6f6b
[ 11.806982] EDAC sbridge: Seeking for: PCI ID 8086:6f6b
[ 11.806987] EDAC sbridge: Seeking for: PCI ID 8086:6f6c
[ 11.807000] EDAC sbridge: Seeking for: PCI ID 8086:6f6d
[ 11.807014] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
[ 11.807019] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6ffc
[ 11.807020] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
[ 11.807027] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6ffc
[ 11.807028] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
[ 11.807034] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
[ 11.807039] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6ffd
[ 11.807041] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
[ 11.807048] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6ffd
[ 11.807049] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
[ 11.807055] EDAC sbridge: Seeking for: PCI ID 8086:6faf
[ 11.807061] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6faf
[ 11.807062] EDAC sbridge: Seeking for: PCI ID 8086:6faf
[ 11.807070] EDAC DEBUG: sbridge_get_onedevice: Detected 8086:6faf
[ 11.807071] EDAC sbridge: Seeking for: PCI ID 8086:6faf
[ 11.807075] EDAC DEBUG: sbridge_probe: Registering MC#0 (1 of 4)
[ 11.807077] EDAC DEBUG: edac_mc_alloc: allocating 2576 bytes for mci data (18 dimms, 18 csrows/channels)
[ 11.807101] EDAC DEBUG: sbridge_register_mci: MC: mci = 000000005210e299, dev = 0000000065b56811
[ 11.807114] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.18.0 with dev = 00000000d243f104
[ 11.807116] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.19.0 with dev = 00000000005f7f1d
[ 11.807118] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.19.1 with dev = 000000006e92822e
[ 11.807119] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.19.2 with dev = 000000000e64b748
[ 11.807120] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.19.3 with dev = 0000000020d7cc1c
[ 11.807122] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.15.4 with dev = 00000000750d55be
[ 11.807123] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.15.5 with dev = 00000000ef303e47
[ 11.807125] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.19.7 with dev = 00000000f9ce7a99
[ 11.807128] EDAC DEBUG: get_dimm_config: mc#0: Node ID: 1, source ID: 1
[ 11.807130] EDAC DEBUG: get_dimm_config: Memory mirroring is disabled
[ 11.807131] EDAC DEBUG: get_dimm_config: Lockstep is disabled
[ 11.807132] EDAC DEBUG: get_dimm_config: address map is on open page mode
[ 11.807134] EDAC DEBUG: __populate_dimms: Memory is registered
[ 11.807136] EDAC DEBUG: __populate_dimms: mc#0: ha 0 channel 0, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807140] EDAC DEBUG: __populate_dimms: mc#0: ha 0 channel 1, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807144] EDAC DEBUG: get_memory_layout: TOLM: 2.000 GB (0x000000007fffffff)
[ 11.807147] EDAC DEBUG: get_memory_layout: TOHM: 66.000 GB (0x000000107fffffff)
[ 11.807149] EDAC DEBUG: get_memory_layout: SAD#0 DRAM up to 34.000 GB (0x0000000880000000) Interleave: [8:6]XOR[18:16] reg=0x040087c1
[ 11.807151] EDAC DEBUG: get_memory_layout: SAD#0, interleave #0: 0
[ 11.807153] EDAC DEBUG: get_memory_layout: SAD#1 DRAM up to 66.000 GB (0x0000001080000000) Interleave: [8:6]XOR[18:16] reg=0x040107c1
[ 11.807155] EDAC DEBUG: get_memory_layout: SAD#1, interleave #0: 1
[ 11.807164] EDAC DEBUG: get_memory_layout: TAD#0: up to 66.000 GB (0x0000001080000000), socket interleave 2, memory interleave 2, TGT: 0, 1, 0, 0, reg=0x0041f504
[ 11.807168] EDAC DEBUG: get_memory_layout: TAD CH#0, offset #0: 34.000 GB (0x0000000880000000), reg=0x00008800
[ 11.807170] EDAC DEBUG: get_memory_layout: TAD CH#1, offset #0: 34.000 GB (0x0000000880000000), reg=0x00008800
[ 11.807171] EDAC DEBUG: get_memory_layout: CH#0 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807174] EDAC DEBUG: get_memory_layout: CH#0 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807177] EDAC DEBUG: get_memory_layout: CH#1 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807179] EDAC DEBUG: get_memory_layout: CH#1 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807181] EDAC DEBUG: edac_mc_add_mc_with_groups:
[ 11.807205] EDAC DEBUG: edac_create_sysfs_mci_device: device mc0 created
[ 11.807223] EDAC DEBUG: edac_create_dimm_object: device dimm0 created at location channel 0 slot 0
[ 11.807240] EDAC DEBUG: edac_create_dimm_object: device dimm3 created at location channel 1 slot 0
[ 11.807261] EDAC DEBUG: edac_create_csrow_object: device csrow0 created
[ 11.807287] EDAC MC0: Giving out device to module sb_edac controller Broadwell SrcID#1_Ha#0: DEV 0000:ff:12.0 (INTERRUPT)
[ 11.807289] EDAC DEBUG: sbridge_probe: Registering MC#1 (2 of 4)
[ 11.807290] EDAC DEBUG: edac_mc_alloc: allocating 2576 bytes for mci data (18 dimms, 18 csrows/channels)
[ 11.807316] EDAC DEBUG: sbridge_register_mci: MC: mci = 00000000b25ea935, dev = 00000000a5576574
[ 11.807329] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.18.0 with dev = 00000000e2584fd1
[ 11.807330] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.19.0 with dev = 000000002a1ac2f9
[ 11.807331] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.19.1 with dev = 000000007dc7650c
[ 11.807333] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.19.2 with dev = 000000006e7b47dc
[ 11.807334] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.19.3 with dev = 00000000cf74f8e4
[ 11.807335] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.15.4 with dev = 0000000095124572
[ 11.807337] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.15.5 with dev = 00000000e9582a11
[ 11.807338] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.19.7 with dev = 000000009207651f
[ 11.807341] EDAC DEBUG: get_dimm_config: mc#1: Node ID: 0, source ID: 0
[ 11.807343] EDAC DEBUG: get_dimm_config: Memory mirroring is disabled
[ 11.807344] EDAC DEBUG: get_dimm_config: Lockstep is disabled
[ 11.807345] EDAC DEBUG: get_dimm_config: address map is on open page mode
[ 11.807346] EDAC DEBUG: __populate_dimms: Memory is registered
[ 11.807348] EDAC DEBUG: __populate_dimms: mc#1: ha 0 channel 0, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807352] EDAC DEBUG: __populate_dimms: mc#1: ha 0 channel 1, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807355] EDAC DEBUG: get_memory_layout: TOLM: 2.000 GB (0x000000007fffffff)
[ 11.807357] EDAC DEBUG: get_memory_layout: TOHM: 66.000 GB (0x000000107fffffff)
[ 11.807359] EDAC DEBUG: get_memory_layout: SAD#0 DRAM up to 34.000 GB (0x0000000880000000) Interleave: [8:6]XOR[18:16] reg=0x040087c1
[ 11.807361] EDAC DEBUG: get_memory_layout: SAD#0, interleave #0: 0
[ 11.807363] EDAC DEBUG: get_memory_layout: SAD#1 DRAM up to 66.000 GB (0x0000001080000000) Interleave: [8:6]XOR[18:16] reg=0x040107c1
[ 11.807364] EDAC DEBUG: get_memory_layout: SAD#1, interleave #0: 1
[ 11.807372] EDAC DEBUG: get_memory_layout: TAD#0: up to 2.000 GB (0x0000000080000000), socket interleave 2, memory interleave 2, TGT: 0, 1, 0, 0, reg=0x0001f504
[ 11.807375] EDAC DEBUG: get_memory_layout: TAD#1: up to 34.000 GB (0x0000000880000000), socket interleave 2, memory interleave 2, TGT: 0, 1, 0, 0, reg=0x0021f504
[ 11.807378] EDAC DEBUG: get_memory_layout: TAD CH#0, offset #0: 0.000 GB (0x0000000000000000), reg=0x00000000
[ 11.807380] EDAC DEBUG: get_memory_layout: TAD CH#0, offset #1: 2.000 GB (0x0000000080000000), reg=0x00000800
[ 11.807382] EDAC DEBUG: get_memory_layout: TAD CH#1, offset #0: 0.000 GB (0x0000000000000000), reg=0x00000000
[ 11.807384] EDAC DEBUG: get_memory_layout: TAD CH#1, offset #1: 2.000 GB (0x0000000080000000), reg=0x00000800
[ 11.807385] EDAC DEBUG: get_memory_layout: CH#0 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807387] EDAC DEBUG: get_memory_layout: CH#0 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807390] EDAC DEBUG: get_memory_layout: CH#1 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807391] EDAC DEBUG: get_memory_layout: CH#1 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807393] EDAC DEBUG: edac_mc_add_mc_with_groups:
[ 11.807417] EDAC DEBUG: edac_create_sysfs_mci_device: device mc1 created
[ 11.807435] EDAC DEBUG: edac_create_dimm_object: device dimm0 created at location channel 0 slot 0
[ 11.807450] EDAC DEBUG: edac_create_dimm_object: device dimm3 created at location channel 1 slot 0
[ 11.807468] EDAC DEBUG: edac_create_csrow_object: device csrow0 created
[ 11.807485] EDAC MC1: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#0: DEV 0000:7f:12.0 (INTERRUPT)
[ 11.807487] EDAC DEBUG: sbridge_probe: Registering MC#2 (3 of 4)
[ 11.807488] EDAC DEBUG: edac_mc_alloc: allocating 2576 bytes for mci data (18 dimms, 18 csrows/channels)
[ 11.807515] EDAC DEBUG: sbridge_register_mci: MC: mci = 0000000067960ead, dev = 000000002e457621
[ 11.807528] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.18.4 with dev = 00000000754ae0a4
[ 11.807529] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.22.0 with dev = 00000000148986d9
[ 11.807530] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.22.1 with dev = 00000000f0d9aa3c
[ 11.807531] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.22.2 with dev = 000000001c2d7f1d
[ 11.807533] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.22.3 with dev = 000000006c26ecab
[ 11.807534] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.15.4 with dev = 00000000750d55be
[ 11.807535] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.15.5 with dev = 00000000ef303e47
[ 11.807537] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI ff.19.7 with dev = 00000000f9ce7a99
[ 11.807539] EDAC DEBUG: get_dimm_config: mc#2: Node ID: 1, source ID: 1
[ 11.807542] EDAC DEBUG: get_dimm_config: Memory mirroring is disabled
[ 11.807543] EDAC DEBUG: get_dimm_config: Lockstep is disabled
[ 11.807544] EDAC DEBUG: get_dimm_config: address map is on open page mode
[ 11.807545] EDAC DEBUG: __populate_dimms: Memory is registered
[ 11.807546] EDAC DEBUG: __populate_dimms: mc#2: ha 1 channel 0, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807550] EDAC DEBUG: __populate_dimms: mc#2: ha 1 channel 1, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807553] EDAC DEBUG: get_memory_layout: TOLM: 2.000 GB (0x000000007fffffff)
[ 11.807556] EDAC DEBUG: get_memory_layout: TOHM: 66.000 GB (0x000000107fffffff)
[ 11.807578] EDAC DEBUG: get_memory_layout: SAD#0 DRAM up to 34.000 GB (0x0000000880000000) Interleave: [8:6]XOR[18:16] reg=0x040087c1
[ 11.807580] EDAC DEBUG: get_memory_layout: SAD#0, interleave #0: 0
[ 11.807582] EDAC DEBUG: get_memory_layout: SAD#1 DRAM up to 66.000 GB (0x0000001080000000) Interleave: [8:6]XOR[18:16] reg=0x040107c1
[ 11.807584] EDAC DEBUG: get_memory_layout: SAD#1, interleave #0: 1
[ 11.807593] EDAC DEBUG: get_memory_layout: TAD#0: up to 66.000 GB (0x0000001080000000), socket interleave 2, memory interleave 2, TGT: 0, 1, 0, 0, reg=0x0041f504
[ 11.807596] EDAC DEBUG: get_memory_layout: TAD CH#0, offset #0: 34.000 GB (0x0000000880000000), reg=0x00008800
[ 11.807598] EDAC DEBUG: get_memory_layout: TAD CH#1, offset #0: 34.000 GB (0x0000000880000000), reg=0x00008800
[ 11.807600] EDAC DEBUG: get_memory_layout: CH#0 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807602] EDAC DEBUG: get_memory_layout: CH#0 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807605] EDAC DEBUG: get_memory_layout: CH#1 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807606] EDAC DEBUG: get_memory_layout: CH#1 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807609] EDAC DEBUG: edac_mc_add_mc_with_groups:
[ 11.807628] EDAC DEBUG: edac_create_sysfs_mci_device: device mc2 created
[ 11.807644] EDAC DEBUG: edac_create_dimm_object: device dimm0 created at location channel 0 slot 0
[ 11.807660] EDAC DEBUG: edac_create_dimm_object: device dimm3 created at location channel 1 slot 0
[ 11.807678] EDAC DEBUG: edac_create_csrow_object: device csrow0 created
[ 11.807697] EDAC MC2: Giving out device to module sb_edac controller Broadwell SrcID#1_Ha#1: DEV 0000:ff:12.4 (INTERRUPT)
[ 11.807698] EDAC DEBUG: sbridge_probe: Registering MC#3 (4 of 4)
[ 11.807699] EDAC DEBUG: edac_mc_alloc: allocating 2576 bytes for mci data (18 dimms, 18 csrows/channels)
[ 11.807725] EDAC DEBUG: sbridge_register_mci: MC: mci = 0000000038d8e473, dev = 0000000072119eb9
[ 11.807738] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.18.4 with dev = 00000000cabb5c8d
[ 11.807739] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.22.0 with dev = 0000000045eb8c12
[ 11.807741] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.22.1 with dev = 0000000015d6baa6
[ 11.807742] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.22.2 with dev = 00000000bb0a15a6
[ 11.807743] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.22.3 with dev = 000000001c5b1eae
[ 11.807745] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.15.4 with dev = 0000000095124572
[ 11.807746] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.15.5 with dev = 00000000e9582a11
[ 11.807747] EDAC DEBUG: broadwell_mci_bind_devs: Associated PCI 7f.19.7 with dev = 000000009207651f
[ 11.807750] EDAC DEBUG: get_dimm_config: mc#3: Node ID: 0, source ID: 0
[ 11.807752] EDAC DEBUG: get_dimm_config: Memory mirroring is disabled
[ 11.807753] EDAC DEBUG: get_dimm_config: Lockstep is disabled
[ 11.807754] EDAC DEBUG: get_dimm_config: address map is on open page mode
[ 11.807755] EDAC DEBUG: __populate_dimms: Memory is registered
[ 11.807757] EDAC DEBUG: __populate_dimms: mc#3: ha 1 channel 0, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807760] EDAC DEBUG: __populate_dimms: mc#3: ha 1 channel 1, dimm 0, 8192 MiB (2097152 pages) bank: 16, rank: 1, row: 0x10000, col: 0x400
[ 11.807764] EDAC DEBUG: get_memory_layout: TOLM: 2.000 GB (0x000000007fffffff)
[ 11.807766] EDAC DEBUG: get_memory_layout: TOHM: 66.000 GB (0x000000107fffffff)
[ 11.807767] EDAC DEBUG: get_memory_layout: SAD#0 DRAM up to 34.000 GB (0x0000000880000000) Interleave: [8:6]XOR[18:16] reg=0x040087c1
[ 11.807769] EDAC DEBUG: get_memory_layout: SAD#0, interleave #0: 0
[ 11.807771] EDAC DEBUG: get_memory_layout: SAD#1 DRAM up to 66.000 GB (0x0000001080000000) Interleave: [8:6]XOR[18:16] reg=0x040107c1
[ 11.807773] EDAC DEBUG: get_memory_layout: SAD#1, interleave #0: 1
[ 11.807781] EDAC DEBUG: get_memory_layout: TAD#0: up to 2.000 GB (0x0000000080000000), socket interleave 2, memory interleave 2, TGT: 0, 1, 0, 0, reg=0x0001f504
[ 11.807783] EDAC DEBUG: get_memory_layout: TAD#1: up to 34.000 GB (0x0000000880000000), socket interleave 2, memory interleave 2, TGT: 0, 1, 0, 0, reg=0x0021f504
[ 11.807786] EDAC DEBUG: get_memory_layout: TAD CH#0, offset #0: 0.000 GB (0x0000000000000000), reg=0x00000000
[ 11.807788] EDAC DEBUG: get_memory_layout: TAD CH#0, offset #1: 2.000 GB (0x0000000080000000), reg=0x00000800
[ 11.807790] EDAC DEBUG: get_memory_layout: TAD CH#1, offset #0: 0.000 GB (0x0000000000000000), reg=0x00000000
[ 11.807792] EDAC DEBUG: get_memory_layout: TAD CH#1, offset #1: 2.000 GB (0x0000000080000000), reg=0x00000800
[ 11.807793] EDAC DEBUG: get_memory_layout: CH#0 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807795] EDAC DEBUG: get_memory_layout: CH#0 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807798] EDAC DEBUG: get_memory_layout: CH#1 RIR#0, limit: 7.999 GB (0x00000001fff00000), way: 1, reg=0x8000001e
[ 11.807799] EDAC DEBUG: get_memory_layout: CH#1 RIR#0 INTL#0, offset 0.000 GB (0x0000000000000000), tgt: 0, reg=0x00000000
[ 11.807802] EDAC DEBUG: edac_mc_add_mc_with_groups:
[ 11.807818] EDAC DEBUG: edac_create_sysfs_mci_device: device mc3 created
[ 11.807834] EDAC DEBUG: edac_create_dimm_object: device dimm0 created at location channel 0 slot 0
[ 11.807850] EDAC DEBUG: edac_create_dimm_object: device dimm3 created at location channel 1 slot 0
[ 11.807867] EDAC DEBUG: edac_create_csrow_object: device csrow0 created
[ 11.807884] EDAC MC3: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#1: DEV 0000:7f:12.4 (INTERRUPT)
[ 11.807886] EDAC sbridge: Ver: 1.1.2
[ 11.825222] intel_rapl_common: Found RAPL domain package
[ 11.825228] intel_rapl_common: Found RAPL domain dram
[ 11.825232] intel_rapl_common: DRAM domain energy unit 15300pj
[ 11.825533] intel_rapl_common: Found RAPL domain package
[ 11.825539] intel_rapl_common: Found RAPL domain dram
[ 11.825543] intel_rapl_common: DRAM domain energy unit 15300pj
[ 12.108120] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[ 12.117234] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[ 12.117450] cfg80211: Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'
[ 12.397573] 8021q: 802.1Q VLAN Support v1.8
[ 12.397587] 8021q: adding VLAN 0 to HW filter on device eth0
[ 12.984138] pps pps1: new PPS source ptp1
[ 12.984191] ixgbe 0000:03:00.1: registered PHC device on eth1
[ 13.107622] 8021q: adding VLAN 0 to HW filter on device eth1
[ 13.910217] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 270.522201] mce: Triggering MCE exception on CPU 0
[ 270.522300] Disabling lock debugging due to kernel taint
[ 270.522390] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 1: b400000000000000
[ 273.358647] mce: [Hardware Error]: TSC 1d6086030ad ADDR 1234
[ 273.365062] mce: [Hardware Error]: PROCESSOR 0:406f0 TIME 1736134149 SOCKET 0 APIC 0 microcode 10
[ 273.374967] mce: [Hardware Error]: Run the above through 'mcelog --ascii'
[ 273.382543] mce: [Hardware Error]: Machine check: MCIP not set in MCA handler
[ 273.390507] Kernel panic - not syncing:
[ 273.394787] Fatal machine check
[ 273.394790] CPU: 44 PID: 0 Comm: swapper/44 Tainted: G M 5.15.169-rt76-yocto-preempt-rt #1
[ 273.408976] Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRRFSDP1.86B.0251.R01.1505050612 05/05/2015
[ 273.420336] Call Trace:
[ 273.423066] <NMI>
[ 273.425312] dump_stack_lvl+0x38/0x4d
[ 273.429407] dump_stack+0x10/0x16
[ 273.433106] panic+0xaa/0x2ed
[ 273.436423] mce_panic+0x175/0x340
[ 273.440224] mce_end+0x20c/0x290
[ 273.443828] do_machine_check+0x601/0x990
[ 273.448305] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.454436] raise_exception+0x39/0xa0
[ 273.458624] ? debug_smp_processor_id+0x17/0x20
[ 273.463685] ? intel_pmu_lbr_enable_all+0x1f/0x1d0
[ 273.469036] ? debug_smp_processor_id+0x17/0x20
[ 273.474088] ? intel_bts_enable_local+0x16/0x60
[ 273.479145] ? x2apic_cluster_probe+0x60/0x60
[ 273.484012] ? intel_pmu_handle_irq+0x25d/0x5a0
[ 273.489070] mce_raise_notify+0xc8/0xe0
[ 273.493354] nmi_handle+0x6f/0x170
[ 273.497153] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.503279] default_do_nmi+0x71/0x200
[ 273.507456] exc_nmi+0xfb/0x130
[ 273.510962] end_repeat_nmi+0x16/0x67
[ 273.515053] RIP: 0010:mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.521857] Code: e2 08 75 26 48 8b 00 a9 00 00 08 00 75 1c 8b 05 14 68 74 01 85 c0 7e 07 0f 00 2d 59 cf d4 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 80 ad 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b
[ 273.542816] RSP: 0018:ffffb34ac659be50 EFLAGS: 00000042
[ 273.548648] RAX: 0000000000000020 RBX: ffffd342bf785500 RCX: 0000000000000001
[ 273.556615] RDX: 0000000000000000 RSI: ffffffffaf3bb220 RDI: 0000000000000020
[ 273.564581] RBP: ffffb34ac659be60 R08: 0000000000030807 R09: 0000000000000018
[ 273.572545] R10: 000000000001ef15 R11: ffff894e5f7a9064 R12: 0000000000000004
[ 273.580509] R13: ffffffffaf3bb220 R14: ffffffffaf3bb3d8 R15: 0000000000000004
[ 273.588478] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.594603] ? mwait_idle_with_hints.constprop.0+0x62/0xc0
[ 273.600719] </NMI>
[ 273.603059] <TASK>
[ 273.605400] ? intel_idle+0x1f/0x30
[ 273.609294] cpuidle_enter_state+0x92/0x3b0
[ 273.613965] cpuidle_enter+0x2e/0x50
[ 273.617954] do_idle+0x22a/0x2b0
[ 273.621564] cpu_startup_entry+0x20/0x30
[ 273.625943] start_secondary+0xf3/0x100
[ 273.630229] secondary_startup_64_no_verify+0xc2/0xcb
[ 273.635871] </TASK>
[ 274.638429] Kernel Offset: 0x2c800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 274.650465] Rebooting in 30 seconds..
