rr icon indicating copy to clipboard operation
rr copied to clipboard

Intermittent counter interrupts missed on AWS Aarch64 Graviton instances

Open rocallahan opened this issue 9 months ago • 28 comments

ESC[?2004h(rr) break abort_ip
break abort_ip
ESC[?2004l^MBreakpoint 1 at 0xaaaacde00ef4
c
ESC[?2004h(rr) c
ESC[?2004l^MContinuing.
[FATAL src/PerfCounters.cc:824:read_ticks()] 
 (task 2569834 (rec:2567804) at time 189)
 -> Assertion `!counting_period || interrupt_val <= adjusted_counting_period' failed to hold. Detected 48306063 ticks, expected no more than 24154018
Tail of trace dump:
...
{
  real_time:1808.840407 global_time:188, event:`SYSCALL: sched_yield' (state:ENTERING_SYSCALL) tid:2567752, ticks:435
x[0]:0x4 x[1]:0xe x[2]:0x0 x[3]:0x0 x[4]:0xffff846f72f0 x[5]:0x64 x[6]:0xfffffff9 x[7]:0x0 x[8]:0x7c x[9]:0xffff8467f861 x[10]:0x0 x[11]:0xa x[12]:0x272e7c x[13]:0x0 x[14]:0x1 x[15]:0xffffffff x[16]:0xaaaacde11f90 x[17]:0xffff84857344 x[18]:0x2 x[19]:0xffffe2e4a288 x[20]:0x1 x[21]:0xaaaacde11d00 x[22]:0xffff848ba040 x[23]:0xaaaacde014f0 x[24]:0xffff8481b000 x[25]:0x0 x[26]:0xffffe2e4a298 x[27]:0xaaaacde11d00 x[28]:0x0 x[29]:0xffff8467ff70 x[30]:0xaaaacde013e4 sp:0xffff8467ff70 pc:0xffff8485735c pstate:0x80001000
}
{
  real_time:1808.900190 global_time:189, event:`SCHED' tid:2567804, ticks:93949855
x[0]:0x0 x[1]:0x4d2 x[2]:0x4d2 x[3]:0xffff848bb038 x[4]:0xffff848bb190 x[5]:0x1 x[6]:0xffff848a9020 x[7]:0xffff84695c00 x[8]:0x63 x[9]:0x30 x[10]:0xaaaacde00fb8 x[11]:0x10f00 x[12]:0x0 x[13]:0x0 x[14]:0x1 x[15]:0xffffffff x[16]:0xaaaacde11f48 x[17]:0xffff84736b10 x[18]:0x2 x[19]:0xffffe2e4a288 x[20]:0x1 x[21]:0xaaaacde11d00 x[22]:0xffff848ba040 x[23]:0xaaaacde014f0 x[24]:0xffff8481b000 x[25]:0x0 x[26]:0xffffe2e4a298 x[27]:0xaaaacde11d00 x[28]:0x0 x[29]:0xffff8467ff80 x[30]:0xaaaacde0133c sp:0xffff8467ff80 pc:0xaaaacde01338 pstate:0x60001000
}
{
  real_time:1808.900300 global_time:190, event:`SYSCALL: sched_yield' (state:EXITING_SYSCALL) tid:2567752, ticks:435
x[0]:0x0 x[1]:0xe x[2]:0x0 x[3]:0x0 x[4]:0xffff846f72f0 x[5]:0x64 x[6]:0xfffffff9 x[7]:0x0 x[8]:0x7c x[9]:0xffff8467f861 x[10]:0x0 x[11]:0xa x[12]:0x272e7c x[13]:0x0 x[14]:0x1 x[15]:0xffffffff x[16]:0xaaaacde11f90 x[17]:0xffff84857344 x[18]:0x2 x[19]:0xffffe2e4a288 x[20]:0x1 x[21]:0xaaaacde11d00 x[22]:0xffff848ba040 x[23]:0xaaaacde014f0 x[24]:0xffff8481b000 x[25]:0x0 x[26]:0xffffe2e4a298 x[27]:0xaaaacde11d00 x[28]:0x0 x[29]:0xffff8467ff70 x[30]:0xaaaacde013e4 sp:0xffff8467ff70 pc:0xffff8485735c pstate:0x80001000
}

rocallahan avatar Sep 17 '23 21:09 rocallahan