rr
rr copied to clipboard
Published
20 hours ago •
rr-debugger
rr-debugger
pkeys test fails on Alder Lake
Continuing from #2997.
I updated to Linux 5.16 release. rr record bin/pkeys + rr replay now gives (on rr master):
(rr) cont
Continuing.
[FATAL ../src/ReplaySession.cc:626:enter_syscall()]
(task 43946 (rec:43943) at time 194)
-> Assertion `false' failed to hold. Expected syscall_bp_vm to be clear but it's 43943's address space with a breakpoint at 0x70000000 while we're at 0x7f907310b9b7
Tail of trace dump:
{
real_time:1297.133446 global_time:174, event:`SYSCALL: prlimit64' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x0 rbx:0x1 rcx:0xffffffffffffffff rdx:0x0 rsi:0x3 rdi:0x0 rbp:0x7ffc4e4aefb0 rsp:0x7ffc4e4aeca8 r8:0x0 r9:0xc r10:0x7ffc4e4aecb0 r11:0x246 r12:0x0 r13:0xfffffffffffffff8 r14:0x7f9073283220 r15:0x7f9073283220 rip:0x7f90731113a4 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12e fs_base:0x7f9072ff1740 gs_base:0x0
{ tid:43943, addr:0x7ffc4e4aecb0, length:0x10 }
}
{
real_time:1297.133546 global_time:175, event:`SYSCALL: munmap' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x7f9073283100 rcx:0xffffffffffffffff rdx:0x7f9073088b00 rsi:0x1cba5 rdi:0x7f907321c000 rbp:0x7ffc4e4aefb0 rsp:0x7ffc4e4aecc8 r8:0x0 r9:0xc r10:0xfffffffffffffb88 r11:0x246 r12:0x0 r13:0xfffffffffffffff8 r14:0x7f9073283220 r15:0x7f9073283220 rip:0x7f907327140b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xb fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.133585 global_time:176, event:`SYSCALL: munmap' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x0 rbx:0x7f9073283100 rcx:0xffffffffffffffff rdx:0x7f9073088b00 rsi:0x1cba5 rdi:0x7f907321c000 rbp:0x7ffc4e4aefb0 rsp:0x7ffc4e4aecc8 r8:0x0 r9:0xc r10:0xfffffffffffffb88 r11:0x246 r12:0x0 r13:0xfffffffffffffff8 r14:0x7f9073283220 r15:0x7f9073283220 rip:0x7f907327140b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xb fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.133687 global_time:177, event:`SYSCALL: rrcall_init_preload' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x7ffc4e4aeeb0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7ffc4e4aee50 rbp:0x7ffc4e4aee50 rsp:0x7ffc4e4aee10 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x7ffc4e4af0d8 r14:0x7f9073243e50 r15:0x0 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e8 fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.133751 global_time:178, event:`SYSCALL: rrcall_init_preload' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x0 rbx:0x7ffc4e4aeeb0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7ffc4e4aee50 rbp:0x7ffc4e4aee50 rsp:0x7ffc4e4aee10 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x7ffc4e4af0d8 r14:0x7f9073243e50 r15:0x0 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e8 fs_base:0x7f9072ff1740 gs_base:0x0
{ tid:43943, addr:0x7f9073244228, length:0x400 }
{ tid:43943, addr:0x7f9073244222, length:0x1 }
{ tid:43943, addr:0x7f9073244224, length:0x4 }
{ tid:43943, addr:0x7f9073244223, length:0x1 }
{ tid:43943, addr:0x7f907324a3e8, length:0x8 }
}
{
real_time:1297.133850 global_time:179, event:`SYSCALL: rrcall_init_preload' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x7ffc4e4aeeb0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7ffc4e4aee50 rbp:0x7ffc4e4aee50 rsp:0x7ffc4e4aee10 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x7ffc4e4af0d8 r14:0x7f9073243e50 r15:0x0 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e8 fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.133908 global_time:180, event:`SYSCALL: rrcall_init_preload' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x0 rbx:0x7ffc4e4aeeb0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7ffc4e4aee50 rbp:0x7ffc4e4aee50 rsp:0x7ffc4e4aee10 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x7ffc4e4af0d8 r14:0x7f9073243e50 r15:0x0 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e8 fs_base:0x7f9072ff1740 gs_base:0x0
{ tid:43943, addr:0x7f9073244228, length:0x400 }
{ tid:43943, addr:0x7f9073244222, length:0x1 }
{ tid:43943, addr:0x7f9073244224, length:0x4 }
{ tid:43943, addr:0x7f9073244223, length:0x1 }
{ tid:43943, addr:0x7f907324a3e8, length:0x8 }
}
{
real_time:1297.134008 global_time:181, event:`INSTRUCTION_TRAP' tid:43943, ticks:0
rax:0x1 rbx:0x2398a7eb rcx:0x98c027bc rdx:0xfc1cc410 rsi:0x0 rdi:0x7 rbp:0x7ffc4e4aef60 rsp:0x7ffc4e4aef58 r8:0x7ffc4e4aef84 r9:0x7f907325ed00 r10:0xfffffffffffff6fe r11:0x4 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x5640284714cd eflags:0x10246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.134174 global_time:182, event:`PATCH_SYSCALL' tid:43943, ticks:0
rax:0x14a rbx:0x0 rcx:0xffffffffffffffff rdx:0x56402847206a rsi:0x0 rdi:0x0 rbp:0x7ffc4e4aefa0 rsp:0x7ffc4e4aef68 r8:0x7ffc4e4aef84 r9:0x7f907325ed00 r10:0xfffffffffffff6fe r11:0x246 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x7f907311dd99 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f9072ff1740 gs_base:0x0
{ map_file:"<ZERO>", addr:0x7f907321d000, length:0x1000, prot_flags:"r-xp", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 }
{ tid:43943, addr:0x7f907321d000, length:0x4f }
{ tid:43943, addr:0x7f907311dd99, length:0x5 }
{ tid:43943, addr:0x7f907311dd9e, length:0x3 }
}
{
real_time:1297.134258 global_time:183, event:`SYSCALL: gettid' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x0 rbp:0x7ffc4e4aefa0 rsp:0x681ffdf0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xba fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.134288 global_time:184, event:`SYSCALL: gettid' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0xaba7 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x0 rbp:0x7ffc4e4aefa0 rsp:0x681ffdf0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xba fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.134379 global_time:185, event:`SYSCALL: perf_event_open' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0xffffffffffffffff rsi:0x0 rdi:0x681ffe60 rbp:0x681ffe60 rsp:0x681ffdf0 r8:0x0 r9:0x0 r10:0xffffffffffffffff r11:0x246 r12:0x7ffc4e4af0c8 r13:0xaba7 r14:0x0 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12a fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.134731 global_time:186, event:`SYSCALL: perf_event_open' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x3 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0xffffffffffffffff rsi:0x0 rdi:0x681ffe60 rbp:0x681ffe60 rsp:0x681ffdf0 r8:0x0 r9:0x0 r10:0xffffffffffffffff r11:0x246 r12:0x7ffc4e4af0c8 r13:0xaba7 r14:0x0 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12a fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.134823 global_time:187, event:`SYSCALL: fcntl' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x64 rsi:0x406 rdi:0x3 rbp:0x681ffe60 rsp:0x681ffd90 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x3 r13:0xaba7 r14:0x0 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x48 fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.134862 global_time:188, event:`SYSCALL: fcntl' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x64 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x64 rsi:0x406 rdi:0x3 rbp:0x681ffe60 rsp:0x681ffd90 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x3 r13:0xaba7 r14:0x0 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x48 fs_base:0x7f9072ff1740 gs_base:0x0
{ tid:43943, addr:0x7f907324428c, length:0x1 }
}
{
real_time:1297.134955 global_time:189, event:`SYSCALL: rrcall_init_buffers' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x681ffe60 rbp:0x681ffe60 rsp:0x681ffdf0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x3 r13:0xaba7 r14:0x64 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e9 fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.135221 global_time:190, event:`SYSCALL: rrcall_init_buffers' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x7f9072ef1000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x681ffe60 rbp:0x681ffe60 rsp:0x681ffdf0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x3 r13:0xaba7 r14:0x64 r15:0x7f9073280c40 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e9 fs_base:0x7f9072ff1740 gs_base:0x0
{ map_file:"<ZERO>", addr:0x7f9072ef1000, length:0x100000, prot_flags:"rw-s", file_offset:0x0, device:64769, inode:131081, data_file:"", data_offset:0x0, file_size:0x100000 }
{ tid:43943, addr:0x7f907324428c, length:0x1 }
{ tid:43943, addr:0x681ffe60, length:0x20 }
}
{
real_time:1297.135328 global_time:191, event:`SYSCALL: pkey_alloc' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x56402847206a rsi:0x0 rdi:0x0 rbp:0x681ffe60 rsp:0x681ffdf0 r8:0x7ffc4e4aef84 r9:0x7f907325ed00 r10:0xfffffffffffff6fe r11:0x246 r12:0x3 r13:0xaba7 r14:0x64 r15:0x7f9073280c40 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x14a fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.135361 global_time:192, event:`SYSCALL: pkey_alloc' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x1 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x56402847206a rsi:0x0 rdi:0x0 rbp:0x681ffe60 rsp:0x681ffdf0 r8:0x7ffc4e4aef84 r9:0x7f907325ed00 r10:0xfffffffffffff6fe r11:0x246 r12:0x3 r13:0xaba7 r14:0x64 r15:0x7f9073280c40 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x14a fs_base:0x7f9072ff1740 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x0 st7:0x0 ymm0:0x0 ymm1:0x101010101010100 ymm2:0x7f907324b530 ymm3:0x5f00 ymm4:0x6c616e7265746e695f666f747274735f ymm5:0x1 ymm6:0x1 ymm7:0x0 ymm8:0x0 ymm9:0x0 ymm10:0x0 ymm11:0x0 ymm12:0x0 ymm13:0x0 ymm14:0x0 ymm15:0x0
}
{
real_time:1297.135498 global_time:193, event:`PATCH_SYSCALL' tid:43943, ticks:0
rax:0x9 rbx:0x22 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x0 rbp:0x0 rsp:0x7ffc4e4aef58 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x7f9073115af4 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f9072ff1740 gs_base:0x0
{ tid:43943, addr:0x7f907321d04f, length:0x4f }
{ tid:43943, addr:0x7f9073115af4, length:0x5 }
{ tid:43943, addr:0x7f9073115af9, length:0x3 }
}
{
real_time:1297.135567 global_time:194, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:43943, ticks:0
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x0 rbp:0x0 rsp:0x681ffdf0 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9072ff1740 gs_base:0x0
}
{
real_time:1297.135611 global_time:195, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:43943, ticks:0
rax:0x7f9073238000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x0 rbp:0x0 rsp:0x681ffdf0 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x7ffc4e4af0c8 r13:0x564028471565 r14:0x0 r15:0x7f9073280c40 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f9072ff1740 gs_base:0x0
{ map_file:"<ZERO>", addr:0x7f9073238000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 }
}
=== Start rr backtrace:
rr(_ZN2rr13dump_rr_stackEv+0x60)[0x55d0547b3298]
rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0x1b2)[0x55d0545b76a2]
rr(+0x3da6fb)[0x55d0545f16fb]
rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x63)[0x55d0545f19a1]
rr(_ZN2rr13ReplaySession13enter_syscallEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x5cb)[0x55d0546ea789]
rr(_ZN2rr13ReplaySession18try_one_trace_stepEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0xfa)[0x55d0546ef618]
rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x1d7)[0x55d0546f0ced]
rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandEl+0x113)[0x55d054710739]
rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0x5b4)[0x55d0545b444c]
rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x561)[0x55d0545b66ff]
rr(+0x4ce4af)[0x55d0546e54af]
rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x41f)[0x55d0546e5fcd]
rr(main+0x278)[0x55d0547ced77]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dfd0)[0x7f5dee872fd0]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x7d)[0x7f5dee87307d]
rr(_start+0x25)[0x55d0545096f5]
=== End rr backtrace
Just to make it clear: this isn't a new failure, it was failing on the older kernel too, but with a different assertion (though also on a different rr master version).
Let's do that, but I'm out of the office the coming week. I'll get back to you.
Same issue with Tiger Lake (i7-1165G7), minimal different trace (but that's with today's master, so this alone may be the reason for it):
[FATAL src/ReplaySession.cc:1156:check_ticks_consistency()]
(task 277946 (rec:277659) at time 191)
-> Assertion `ticks_now == trace_ticks' failed to hold. ticks mismatch for 'PATCH_SYSCALL'; expected 28312, got 28668
Tail of trace dump:
{
real_time:26186.465248 global_time:171, event:`SYSCALL: mprotect' (state:EXITING_SYSCALL) tid:277659, ticks:26997
rax:0x0 rbx:0x0 rcx:0xffffffffffffffff rdx:0x1 rsi:0x2000 rdi:0x7f321dd55000 rbp:0x7ffdb50ead70 rsp:0x7ffdb50eac68 r8:0x7f321dad5e60 r9:0x0 r10:0x7f321dd1d000 r11:0x246 r12:0x0 r13:0x0 r14:0x7f321dd57af0 r15:0x7f321dd57af0 rip:0x7f321dd43d4b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xa fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465279 global_time:172, event:`INSTRUCTION_TRAP' tid:277659, ticks:26999
rax:0xfa174b9b rbx:0xd08a6fa12fd88 rcx:0xffffffffffffffff rdx:0xd08a6 rsi:0x2000 rdi:0x7f321dd55000 rbp:0x7ffdb50eb050 rsp:0x7ffdb50ead80 r8:0x7f321dad5e60 r9:0x0 r10:0x7f321dd1d000 r11:0x246 r12:0x7f321dd582e0 r13:0xfffffffffffffff8 r14:0x7f321dacb740 r15:0x7f321dd582e0 rip:0x7f321dd41232 eflags:0x10246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465321 global_time:173, event:`SYSCALL: prlimit64' (state:ENTERING_SYSCALL) tid:277659, ticks:27022
rax:0xffffffffffffffda rbx:0x1 rcx:0xffffffffffffffff rdx:0x0 rsi:0x3 rdi:0x0 rbp:0x7ffdb50eb050 rsp:0x7ffdb50ead48 r8:0x0 r9:0xf r10:0x7ffdb50ead50 r11:0x246 r12:0x7f321dd582e0 r13:0xfffffffffffffff8 r14:0x7f321dacb740 r15:0x7f321dd582e0 rip:0x7f321dbe8494 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12e fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465351 global_time:174, event:`SYSCALL: prlimit64' (state:EXITING_SYSCALL) tid:277659, ticks:27022
rax:0x0 rbx:0x1 rcx:0xffffffffffffffff rdx:0x0 rsi:0x3 rdi:0x0 rbp:0x7ffdb50eb050 rsp:0x7ffdb50ead48 r8:0x0 r9:0xf r10:0x7ffdb50ead50 r11:0x246 r12:0x7f321dd582e0 r13:0xfffffffffffffff8 r14:0x7f321dacb740 r15:0x7f321dd582e0 rip:0x7f321dbe8494 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12e fs_base:0x7f321dacb740 gs_base:0x0
{ tid:277659, addr:0x7ffdb50ead50, length:0x10 }
}
{
real_time:26186.465392 global_time:175, event:`SYSCALL: munmap' (state:ENTERING_SYSCALL) tid:277659, ticks:27157
rax:0xffffffffffffffda rbx:0x7f321dd58118 rcx:0xffffffffffffffff rdx:0x7f321db5ec90 rsi:0x12d99 rdi:0x7f321dcf6000 rbp:0x7ffdb50eb050 rsp:0x7ffdb50ead68 r8:0x0 r9:0xf r10:0x7f321dd1d860 r11:0x246 r12:0x7f321dd582e0 r13:0xfffffffffffffff8 r14:0x7f321dacb740 r15:0x7f321dd582e0 rip:0x7f321dd43d1b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xb fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465430 global_time:176, event:`SYSCALL: munmap' (state:EXITING_SYSCALL) tid:277659, ticks:27157
rax:0x0 rbx:0x7f321dd58118 rcx:0xffffffffffffffff rdx:0x7f321db5ec90 rsi:0x12d99 rdi:0x7f321dcf6000 rbp:0x7ffdb50eb050 rsp:0x7ffdb50ead68 r8:0x0 r9:0xf r10:0x7f321dd1d860 r11:0x246 r12:0x7f321dd582e0 r13:0xfffffffffffffff8 r14:0x7f321dacb740 r15:0x7f321dd582e0 rip:0x7f321dd43d1b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xb fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465471 global_time:177, event:`SYSCALL: rrcall_init_preload' (state:ENTERING_SYSCALL) tid:277659, ticks:27423
rax:0xffffffffffffffda rbx:0x7ffdb50eaf90 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7ffdb50eaf30 rbp:0x1 rsp:0x7ffdb50eaef0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x7ffdb50eb1d8 r14:0x7f321dd13e50 r15:0x0 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e8 fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465522 global_time:178, event:`SYSCALL: rrcall_init_preload' (state:EXITING_SYSCALL) tid:277659, ticks:27423
rax:0x0 rbx:0x7ffdb50eaf90 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x7ffdb50eaf30 rbp:0x1 rsp:0x7ffdb50eaef0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x7ffdb50eb1d8 r14:0x7f321dd13e50 r15:0x0 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e8 fs_base:0x7f321dacb740 gs_base:0x0
{ tid:277659, addr:0x7f321dd14228, length:0x400 }
{ tid:277659, addr:0x7f321dd14222, length:0x1 }
{ tid:277659, addr:0x7f321dd14224, length:0x4 }
{ tid:277659, addr:0x7f321dd14223, length:0x1 }
{ tid:277659, addr:0x7f321dd1a3e8, length:0x8 }
}
{
real_time:26186.465559 global_time:179, event:`INSTRUCTION_TRAP' tid:277659, ticks:28284
rax:0x0 rbx:0xf3bba7eb rcx:0x18c05fde rdx:0xfc100510 rsi:0x0 rdi:0x7 rbp:0x7ffdb50eb070 rsp:0x7ffdb50eb068 r8:0x7ffdb50eb094 r9:0x7f321dd23040 r10:0x7f321dd1d908 r11:0x7f321dd38680 r12:0x7ffdb50eb1c8 r13:0x561aa76cb50d r14:0x561aa76cdd78 r15:0x7f321dd57040 rip:0x561aa76cb46a eflags:0x10246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465662 global_time:180, event:`PATCH_SYSCALL' tid:277659, ticks:28286
rax:0x14a rbx:0x0 rcx:0xffffffffffffffff rdx:0x35 rsi:0x0 rdi:0x0 rbp:0x7ffdb50eb0b0 rsp:0x7ffdb50eb078 r8:0x7f321dd23040 r9:0x0 r10:0x7ffdb50eb094 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x561aa76cb50d r14:0x561aa76cdd78 r15:0x7f321dd57040 rip:0x7f321dbeca3b eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f321dacb740 gs_base:0x0
{ map_file:"<ZERO>", addr:0x7f321dcf7000, length:0x1000, prot_flags:"r-xp", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 }
{ tid:277659, addr:0x7f321dcf7000, length:0x4f }
{ tid:277659, addr:0x7f321dbeca3b, length:0x8 }
}
{
real_time:26186.465700 global_time:181, event:`SYSCALL: gettid' (state:ENTERING_SYSCALL) tid:277659, ticks:28290
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x0 rbp:0x7ffdb50eb0b0 rsp:0x681ffe20 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x561aa76cb50d r14:0x561aa76cdd78 r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xba fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465723 global_time:182, event:`SYSCALL: gettid' (state:EXITING_SYSCALL) tid:277659, ticks:28290
rax:0x43c9b rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x0 rbp:0x7ffdb50eb0b0 rsp:0x681ffe20 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x561aa76cb50d r14:0x561aa76cdd78 r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xba fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465752 global_time:183, event:`SYSCALL: perf_event_open' (state:ENTERING_SYSCALL) tid:277659, ticks:28290
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0xffffffffffffffff rsi:0x0 rdi:0x681ffe70 rbp:0x7ffdb50eb0b0 rsp:0x681ffe20 r8:0x0 r9:0x0 r10:0xffffffffffffffff r11:0x246 r12:0x7ffdb50eb1c8 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12a fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465784 global_time:184, event:`SYSCALL: perf_event_open' (state:EXITING_SYSCALL) tid:277659, ticks:28290
rax:0x3 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0xffffffffffffffff rsi:0x0 rdi:0x681ffe70 rbp:0x7ffdb50eb0b0 rsp:0x681ffe20 r8:0x0 r9:0x0 r10:0xffffffffffffffff r11:0x246 r12:0x7ffdb50eb1c8 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x12a fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465818 global_time:185, event:`SYSCALL: fcntl' (state:ENTERING_SYSCALL) tid:277659, ticks:28291
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x64 rsi:0x406 rdi:0x3 rbp:0x3 rsp:0x681ffdc0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x48 fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.465851 global_time:186, event:`SYSCALL: fcntl' (state:EXITING_SYSCALL) tid:277659, ticks:28291
rax:0x64 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x64 rsi:0x406 rdi:0x3 rbp:0x3 rsp:0x681ffdc0 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x48 fs_base:0x7f321dacb740 gs_base:0x0
{ tid:277659, addr:0x7f321dd1428c, length:0x1 }
}
{
real_time:26186.465887 global_time:187, event:`SYSCALL: rrcall_init_buffers' (state:ENTERING_SYSCALL) tid:277659, ticks:28296
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x681ffe70 rbp:0x3 rsp:0x681ffe20 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x64 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e9 fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.466143 global_time:188, event:`SYSCALL: rrcall_init_buffers' (state:EXITING_SYSCALL) tid:277659, ticks:28296
rax:0x7f321d9cb000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x0 rdi:0x681ffe70 rbp:0x3 rsp:0x681ffe20 r8:0x0 r9:0x0 r10:0x0 r11:0x246 r12:0x64 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000005 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3e9 fs_base:0x7f321dacb740 gs_base:0x0
{ map_file:"<ZERO>", addr:0x7f321d9cb000, length:0x100000, prot_flags:"rw-s", file_offset:0x0, device:66309, inode:6819434, data_file:"", data_offset:0x0, file_size:0x100000 }
{ tid:277659, addr:0x7f321dd1428c, length:0x1 }
{ tid:277659, addr:0x681ffe70, length:0x20 }
}
{
real_time:26186.466186 global_time:189, event:`SYSCALL: pkey_alloc' (state:ENTERING_SYSCALL) tid:277659, ticks:28302
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x35 rsi:0x0 rdi:0x0 rbp:0x681fffa0 rsp:0x681ffd60 r8:0x7f321dd23040 r9:0x0 r10:0x7ffdb50eb094 r11:0x246 r12:0x64 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x14a fs_base:0x7f321dacb740 gs_base:0x0
}
{
real_time:26186.466215 global_time:190, event:`SYSCALL: pkey_alloc' (state:EXITING_SYSCALL) tid:277659, ticks:28302
rax:0x1 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x35 rsi:0x0 rdi:0x0 rbp:0x681fffa0 rsp:0x681ffd60 r8:0x7f321dd23040 r9:0x0 r10:0x7ffdb50eb094 r11:0x246 r12:0x64 r13:0x681ffe70 r14:0x43c9b r15:0x7f321dd57040 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x14a fs_base:0x7f321dacb740 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x0 st7:0x0 ymm0:0x0 ymm1:0x0 ymm2:0x0 ymm3:0x6b636f ymm4:0x616e7265746e695f666f747274735f5f ymm5:0xd98 ymm6:0x60 ymm7:0x1 ymm8:0x0 ymm9:0x0 ymm10:0x0 ymm11:0x0 ymm12:0x0 ymm13:0x0 ymm14:0x0 ymm15:0x0
}
{
real_time:26186.466286 global_time:191, event:`PATCH_SYSCALL' tid:277659, ticks:28312
rax:0x9 rbx:0x0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x0 rbp:0x7ffdb50eb0b0 rsp:0x7ffdb50eb078 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x561aa76cb50d r14:0x561aa76cdd78 r15:0x7f321dd57040 rip:0x7f321dbecbd5 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7f321dacb740 gs_base:0x0
{ tid:277659, addr:0x7f321dcf704f, length:0x4f }
{ tid:277659, addr:0x7f321dbecbd5, length:0x8 }
}
{
real_time:26186.466320 global_time:192, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:277659, ticks:28320
rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x0 rbp:0x681fffa0 rsp:0x681ffd60 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x7ffdb50eb1c8 r13:0x561aa76cb50d r14:0x561aa76cdd78 r15:0x7f321dd57040 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7f321dacb740 gs_base:0x0
}
=== Start rr backtrace:
rr(_ZN2rr13dump_rr_stackEv+0x60)[0x55b37bdc1465]
rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0xfb)[0x55b37bbddf0f]
rr(+0x12ab26)[0x55b37bc14b26]
rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x63)[0x55b37bb31439]
rr(_ZN2rr13ReplaySession23check_ticks_consistencyEPNS_10ReplayTaskERKNS_5EventE+0x143)[0x55b37bd01611]
rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x8a4)[0x55b37bd059f0]
rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandEl+0xed)[0x55b37bd247a9]
rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0x68b)[0x55b37bbdab9b]
rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x567)[0x55b37bbdd023]
rr(+0x210768)[0x55b37bcfa768]
rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x43a)[0x55b37bcfb2a2]
rr(main+0x278)[0x55b37bdcda44]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f7649595d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f7649595e40]
rr(_start+0x25)[0x55b37bb1e885]
gdbserver state:
~$ gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:16965' /home/simon/.local/share/rr/pkeys-0/mmap_hardlink_4_pkeys
GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/simon/.local/share/rr/pkeys-0/mmap_hardlink_4_pkeys...
Remote debugging using 127.0.0.1:16965
Reading symbols from /usr/local/bin/../lib/rr/librrpreload.so...
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
Reading symbols from /usr/lib/debug/.build-id/89/c3cb85f9e55046776471fed05ec441581d1969.debug...
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/aa/1b0b998999c397062e1016f0c95dc0e8820117.debug...
BFD: warning: system-supplied DSO at 0x6fffd000 has a section extending past end of file
0x00007f321dbe2a35 in __GI___libc_write (fd=1, buf=0x7ffdb50eab70, nbytes=128) at ../sysdeps/unix/sysv/linux/write.c:26
26 ../sysdeps/unix/sysv/linux/write.c: File not found.
(gdb) thread apply all bt
Thread 1 (Thread 277659.277659 (pkeys)):
#0 0x00007f321dbe2a35 in __GI___libc_write (fd=1, buf=0x7ffdb50eab70, nbytes=128) at ../sysdeps/unix/sysv/linux/write.c:26
#1 0x0000561aa76cb346 in atomic_printf (fmt=0x561aa76cc010 "FAILED at %s:%d: !(%s) errno:%d (%s)\n") at /home/install/rr/src/test/x86/../util.h:168
#2 0x0000561aa76cb3e5 in atomic_assert (cond=0, str=0x561aa76cc128 "(initial_pkru & ~(PKEY_DISABLE_ACCESS << (2 * pkey ))) == modified_pkru", file=0x561aa76cc072 "src/test/x86/pkeys.c", line=69) at /home/install/rr/src/test/x86/../util.h:187
#3 0x0000561aa76cb6e4 in main () at /home/install/rr/src/test/x86/pkeys.c:69
(gdb)
Running on kernel 5.15.0-37-generic.
What kernel version?
Sorry for this missing piece - up to date Ubuntu on that machine kernel 5.15.0-37-generic.
I am encountering similar failures as well. 5.15.0-41-generic here.
taskset -c 0-15 ninja test
99% tests passed, 9 tests failed out of 2825
Total Test time (real) = 1777.13 sec
The following tests FAILED:
456 - x86/pkeys (Failed)
457 - x86/pkeys-no-syscallbuf (Failed)
1048 - nested_detach_wait (Failed)
1295 - nested_detach_kill-no-syscallbuf (Failed)
1528 - detach_sigkill_exit-32 (Failed)
1868 - x86/pkeys-32 (Failed)
1869 - x86/pkeys-32-no-syscallbuf (Failed)
2460 - nested_detach_wait-32 (Failed)
2461 - nested_detach_wait-32-no-syscallbuf (Failed)
The pkeys issues should be fixed when https://lkml.org/lkml/2022/11/15/1402 makes its way to the kernel.
And the nested_detach issues on Alder Lake are probably fixed by the diff in #3402
Alright the kernel fix is now in the queue for every supported version. It'll take time to propagate to distro kernels but we're done.