stackcount failure

[protik77]

Also I am trying the code listing in page 101 of the book "BPF Performance Tools". The code is,

stackcount -f -P -D 10 ktime_get > out.stackcount01.txt

A file named out.stackcount01.txt is created but it reads,

sequence item 1: expected str instance, bytes found

The OS is a SLES 15 SP1 with kernel version 4.12.14-197.34-default. The Python version is 3.6.10.

[yonghong-song]

This probably a python3 comparability issue. Could you help debug and submit a pull request?

[protik77]

I would love to, but I still a newbie by all means. I will try to fix it and submit a pull request if I can resolve the issue.

[devidasjadhav]

@protik77

1. can you please try without -f option just to narrow down the issue.

if that doesn't give the error then 2. can you try this stack counr version on my branch

this has diff

@@ -356,7 +356,7 @@ class Tool(object):
                         reversed(user_stack)] + \
                         (self.need_delimiter and ["-"] or []) + \
                         [b.ksym(addr).decode('utf-8', 'replace') for addr in reversed(kernel_stack)]
-                    print("%s %d" % (";".join(line), v.value))
+                    print("%s %d" % (";".join(str(line)), v.value))
                 else:
                     # print multi-line stack output
                     for addr in kernel_stack:

if you confirm I an create pull request.

Explanation

lets start with your error

sequence item 1: expected str instance, bytes found

is type mismatch error mostly this error occurs in .join()

item 1 indicates its argument passed to join is issue. ie, in our case line is its line

although this is just patch not addressing real issue why line may have byte type. may be you can print and check whats exact type and value of line in your case.

[protik77]

1. Yes, removing -f option provides the right output.
2. I tried the stackcount branch, but it puts ; between every character. For example,

[;';d;o;c;k;e;r;-;c;o;n;t;a;i;n;e;';,; ;';r;u;n;t;i;m;e;.;g;o;e;x;i;t;';,; ;';g;i;t;h;u;b;.;c;o;m;/;c;o;n;t;a;i;n;e;r;d;/;c;o;n;t;a;i;n;e;r;d;/;v;e;n;d;o;r;/;g;o;o;g;l;e;.;g;o;l;a;n;g;.;o;r;g;/;g;r;p;c;/;i;n;t;e;r;n;a;l;/;t;r;a;n;s;p;o;r;t;.;n;e;w;H;T;T;P;2;S;e;r;v;e;r;.;f;u;n;c;2;';,; ;';g;i;t;h;u;b;.;c;o;m;/;c;o;n;t;a;i;n;e;r;d;/;c;o;n;t;a;i;n;e;r;d;/;v;e;n;d;o;r;/;g;o;o;g;l;e;.;g;o;l;a;n;g;.;o;r;g;/;g;r;p;c;/;i;n;t;e;r;n;a;l;/;t;r;a;n;s;p;o;r;t;.;(;*;l;o;o;p;y;W;r;i;t;e;r;);.;r;u;n;';,; ;';g;i;t;h;u;b;.;c;o;m;/;c;o;n;t;a;i;n;e;r;d;/;c;o;n;t;a;i;n;e;r;d;/;v;e;n;d;o;r;/;g;o;o;g;l;e;.;g;o;l;a;n;g;.;o;r;g;/;g;r;p;c;/;i;n;t;e;r;n;a;l;/;t;r;a;n;s;p;o;r;t;.;(;*;b;u;f;W;r;i;t;e;r;);.;F;l;u;s;h;';,; ;';n;e;t;.;(;*;c;o;n;n;);.;W;r;i;t;e;';,; ;';n;e;t;.;(;*;n;e;t;F;D;);.;W;r;i;t;e;';,; ;';i;n;t;e;r;n;a;l;/;p;o;l;l;.;(;*;F;D;);.;W;r;i;t;e;';,; ;';i;n;t;e;r;n;a;l;/;p;o;l;l;.;(;*;F;D;);.;w;r;i;t;e;U;n;l;o;c;k;';,; ;';i;n;t;e;r;n;a;l;/;p;o;l;l;.;(;*;f;d;M;u;t;e;x;);.;r;w;u;n;l;o;c;k;';] 1
[;';n;o;d;e;_;e;x;p;o;r;t;e;r;';,; ;';e;n;t;r;y;_;S;Y;S;C;A;L;L;_;6;4;_;a;f;t;e;r;_;h;w;f;r;a;m;e;';,; ;';d;o;_;s;y;s;c;a;l;l;_;6;4;';,; ;';S;y;S;_;f;u;t;e;x;';,; ;';d;o;_;f;u;t;e;x;';,; ;';f;u;t;e;x;_;w;a;i;t;';,; ;';f;u;t;e;x;_;w;a;i;t;_;q;u;e;u;e;_;m;e;';,; ;';p;l;i;s;t;_;a;d;d;';,; ;';a;p;i;c;_;t;i;m;e;r;_;i;n;t;e;r;r;u;p;t;';,; ;';s;m;p;_;a;p;i;c;_;t;i;m;e;r;_;i;n;t;e;r;r;u;p;t;';,; ;';h;r;t;i;m;e;r;_;i;n;t;e;r;r;u;p;t;';,; ;';_;_;h;r;t;i;m;e;r;_;r;u;n;_;q;u;e;u;e;s;';,; ;';t;i;c;k;_;s;c;h;e;d;_;t;i;m;e;r;';,; ;';k;t;i;m;e;_;g;e;t;';] 1
[;';n;o;d;e;_;e;x;p;o;r;t;e;r;';,; ;';[;u;n;k;n;o;w;n;];';,; ;';[;u;n;k;n;o;w;n;];';,; ;';i;n;t;e;r;n;a;l;/;p;o;l;l;.;(;*;F;D;);.;C;l;o;s;e;';,; ;';i;n;t;e;r;n;a;l;/;p;o;l;l;.;r;u;n;t;i;m;e;_;S;e;m;a;c;q;u;i;r;e;';,; ;';r;u;n;t;i;m;e;.;m;c;a;l;l;';,; ;';r;u;n;t;i;m;e;.;e;x;i;t;s;y;s;c;a;l;l;0;';,; ;';r;u;n;t;i;m;e;.;s;c;h;e;d;u;l;e;';,; ;';r;u;n;t;i;m;e;.;f;i;n;d;r;u;n;n;a;b;l;e;';,; ;';r;u;n;t;i;m;e;.;r;u;n;q;s;t;e;a;l;';,; ;';r;u;n;t;i;m;e;.;r;u;n;q;g;r;a;b;';,; ;';r;u;n;t;i;m;e;.;u;s;l;e;e;p;';,; ;';e;n;t;r;y;_;S;Y;S;C;A;L;L;_;6;4;_;a;f;t;e;r;_;h;w;f;r;a;m;e;';,; ;';d;o;_;s;y;s;c;a;l;l;_;6;4;';,; ;';s;y;s;_;n;a;n;o;s;l;e;e;p;';,; ;';h;r;t;i;m;e;r;_;n;a;n;o;s;l;e;e;p;';,; ;';d;o;_;n;a;n;o;s;l;e;e;p;';,; ;';h;r;t;i;m;e;r;_;s;t;a;r;t;_;r;a;n;g;e;_;n;s;';,; ;';c;l;o;c;k;e;v;e;n;t;s;_;p;r;o;g;r;a;m;_;e;v;e;n;t;';,; ;';k;t;i;m;e;_;g;e;t;';] 1

[devidasjadhav]

thanks. here is work temporary around.

diff --git a/tools/stackcount.py b/tools/stackcount.py
index 1691ef68..2e811ef2 100755
--- a/tools/stackcount.py
+++ b/tools/stackcount.py
@@ -356,7 +356,7 @@ class Tool(object):
                         reversed(user_stack)] + \
                         (self.need_delimiter and ["-"] or []) + \
                         [b.ksym(addr).decode('utf-8', 'replace') for addr in reversed(kernel_stack)]
-                    print("%s %d" % (";".join(line), v.value))
+                    print("%s %d" % (b"".join(str(line)), v.value))
                 else:
                     # print multi-line stack output
                     for addr in kernel_stack:

output

$ sudo ./tools/stackcount.py -f -P -D 10 ktime_get
[u'DOM Worker', u'[unknown]', u'[unknown]', u'[unknown]', u'[unknown]', u'entry_SYSCALL_64_after_hwframe', u'do_syscall_64', u'syscall_trace_enter', u'__seccomp_filter', u'__bpf_prog_run32', u'___bpf_prog_run', u'apic_timer_interrupt', u'smp_apic_timer_interrupt', u'hrtimer_interrupt', u'__hrtimer_run_queues', u'tick_sched_timer', u'ktime_get'] 1
[u'Timer', u'[unknown]', u'[unknown]', u'entry_SYSCALL_64_after_hwframe', u'do_syscall_64', u'ksys_write', u'vfs_write', u'new_sync_write', u'pipe_write', u'__wake_up_common_lock', u'_raw_spin_unlock_irqrestore', u'apic_timer_interrupt', u'smp_apic_timer_interrupt', u'hrtimer_interrupt', u'__hrtimer_run_queues', u'tick_sched_timer', u'ktime_get'] 1
[u'swapper/2', u'secondary_startup_64', u'start_secondary', u'cpu_startup_entry', u'do_idle', u'cpuidle_enter', u'cpuidle_enter_state', u'ret_from_intr', u'do_IRQ', u'irq_exit', u'__softirqentry_text_start', u'blk_done_softirq', u'scsi_io_completion', u'scsi_end_request', u'mq_flush_data_end_io', u'blk_flush_complete_seq', u'blk_rq_init', u'ktime_get'] 1

need to figure out why we receive single characters instead of line.

[devidasjadhav]

root-cause

in print("%s %d" % (";".join(line), v.value)) line is expected to be list of 'str' type.

so with following patch we are ensuring any non string char is converted to string. this is preventing probable errors mentioned in this issue.

--- a/tools/stackcount.py
+++ b/tools/stackcount.py
@@ -356,7 +356,7 @@ class Tool(object):
                         reversed(user_stack)] + \
                         (self.need_delimiter and ["-"] or []) + \
                         [b.ksym(addr).decode('utf-8', 'replace') for addr in reversed(kernel_stack)]
-                    print("%s %d" % (";".join(line), v.value))
+                    print("%s %d" % (b";".join([str(word) for word in line]), v.value))
                 else:
                     # print multi-line stack output
                     for addr in kernel_stack:

@protik77 can you try stackcount from strcnt branch it has above diff applied.

[protik77]

I tried the strcnt branch one, now the out.stackcount01.txt file reads,

sequence item 0: expected a bytes-like object, str found

[devidasjadhav]

I have changed code. cause of this was b before `";". from python bug 24892

>>> x = b'foo'
>>> y = b'barbaz'
>>> x.join(y)
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    x.join(y)
TypeError: sequence item 0: expected a bytes-like object, int found

[protik77]

Now the output looks like this,

swapper/10;secondary_startup_64;start_secondary;cpu_startup_entry;do_idle;tick_nohz_idle_exit;apic_timer_interrupt;smp_apic_timer_interrupt;hrtimer_interrupt;clockevents_program_event;ktime_get 1
prometheus;runtime.goexit;github.com/prometheus/prometheus/scrape.(*scrapeLoop).run;runtime.selectgo;runtime.mcall;runtime.park_m;runtime.schedule;runtime.findrunnable;runtime.runqsteal;runtime.runqgrab;runtime.usleep;entry_SYSCALL_64_after_hwframe;do_syscall_64;sys_nanosleep;hrtimer_nanosleep;do_nanosleep;hrtimer_start_range_ns;ktime_get 1
swapper/2;secondary_startup_64;start_secondary;cpu_startup_entry;do_idle;cpuidle_enter_state;apic_timer_interrupt;smp_apic_timer_interrupt;irq_exit;__tick_nohz_idle_enter;tick_nohz_stop_sched_tick;hrtimer_start_range_ns;clockevents_program_event;ktime_get 1
git-remote-http;[unknown];apic_timer_interrupt;smp_apic_timer_interrupt;hrtimer_interrupt;clockevents_program_event;ktime_get 1

[devidasjadhav]

correct now you can use flamegraph. if it works close the issue

[ami-GS]

Ubuntu 20.04 LTS with python 3.8.10 (default python3) faces related issue. Ubuntu 22.04 LTS and Ubuntu 24.04 LTS doesn't

$ sudo /sbin/profile-bpfcc -afd -p pgrep $MY_PROCESS --stack-storage-size 2097152 > profile.stacks

Traceback (most recent call last): File "/sbin/profile-bpfcc", line 342, in print("%s %d" % (b";".join(line).decode('utf-8', 'replace'), v.value)) TypeError: sequence item 10: expected a bytes-like object, str found