offcputime misses the user stack

[dreamedcheng]

I run a program below, and use offcputime to trace the off-cpu stack, but it misses the user stack.

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>

void f(void)
{
    int i;

    for (i = 0; i < 5; i++)
    {
        sleep(5);
    }
}

void g(void)
{
    f();
}

int main()
{
    g();
    return 0;
}

The code is compiled by gcc -O0 -g -fno-omit-frame-pointer test.c, gcc version is 7.3.

The trace command is ./bcc-bin/share/bcc/tools/offcputime -f --stack-storage-size 1024 -p pgrep a.out

The output of the trace is

a.out;__nanosleep_nocancel;entry_SYSCALL_64_after_hwframe;do_syscall_64;__x64_sys_nanosleep;hrtimer_nanosleep;do_nanosleep;schedule;finish_task_switch 6000195

but I think the correct full stack should be a.out;g;f;__nanosleep_nocancel;...

My kernel version is 5.4.5-1.el7.elrepo.x86_64 (Centos 7), bcc is compiled using latest master branch code.

What can I do to show the full stack of off-cpu time?

[shiyiyuedeyu]

Have this problem solved? The output of offcputime: finish_task_switch schedule schedule_hrtimeout_range_clock schedule_hrtimeout_range ep_poll do_epoll_wait __x64_sys_epoll_wait do_syscall_64 entry_SYSCALL_64_after_hwframe [unknown] [unknown] [unknown]

[yonghong-song]

There are a couple of reasons. The process is already gone or the user bottom stack (like libc) does not preserve frame pointers. You can see an example of wakeuptime.py to print out the virtual address if your process is still live at the time of symbol resolution and stack dump.

[baiwfg2]

@yonghong-song I met with same problem in kernel 5.15 version.

But I try it on kernel 4.19, it'll be ok. I can see the whole user stack @dreamedcheng . Maybe you should have a try.

So the question is why not working in higher kernel version like 5.15 ?

[lygn128]

also have no user stack ubuntn 20.04, and kernel is: Linux ubuntu 5.15.0-67-generic #74~20.04.1-Ubuntu SMP Wed Feb 22 14:52:34 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[TheEaterr]

I also have no user stack in ubuntu 22.04 and kernel: Linux 5.15.0-112-generic #122-Ubuntu SMP Thu May 23 07:48:21 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

[yunzhao-cf]

Same issue for no user stack, and we are running kernel 6.6. Could offcputime add "--call-graph" option support?