bcc
bcc copied to clipboard
iovisor
High percentage of Missed User Stacks profile tool with chrome browser
When running chrome (web browser built form sources with support for profiling/perf to do stack unwinding etc ) on ubuntu18.04.1 LTS and kernel 4.15.0-44-generic x86_64 sudo /usr/share/bcc/tools/profile -f -F 1000 -p PID-of-chrome > chrome.stacks Then loading and starting the jetstream https://browserbench.org/JetStream/ javascript benchmark (clicking on test) and running for 1 iteration prior to CTRL-C of the profile I get a large percentage - typically around 20% and sometimes as high as 56% of total stacks as "Missed User Stack". Im running on a standard lenovo desktop 4 real cores.
NOTE: perf does not suffer from this problem. I've tried running with/without perf jit symbol support. altering the --stack-storage-size and they don't make a difference. I use top to determine the correct TGID to use for -p. Such a large error makes any generated flamegraphs pretty unusable. Have I missed anything obvious here?
Hi, ok running for a timed session of 50s, and massively increasing --stack-storage-size leads to an acceptable reduction in Missed User Stacks ... this is unusual as I do not get any ENOMEM related messages so something strange is happening - I don't think this is an issue related to chrome NOTE - as root after increasing various ulimits .... /usr/share/bcc/tools/profile -f -F 1000 --stack-storage-size 2024000 -p PID-of-chrome 50 > chrome.stacks 1KHz sampling is not stressing the system too much, Ive turned hyper threading off to see if that makes a difference. A total of 10240 unique stacks were stored for this execution of chrome. I have a hacked version of profile that opens /tmp/perf-PID.map to resolve jit-ted symbol addresses.
Hmm some of the stacks in the bcc profile generated flamegraph are very very deep, I didnt see these with perf but I may have sampled a different part of the jetstream tests in execution ....
Hi, ok the very very large stack depths were seemingly caused by incorrect processing by the flamegraph perl script ... lots of regex type boxes appearing... note we have symbols using the wildcard character LazyCompile* and the javascript_filename.js:245 for the source javascript file and line number appearing. Renaming the symbols to javascript_filename.js_245 seems to do the trick for the flamegraph.
