kubectl-flame icon indicating copy to clipboard operation
kubectl-flame copied to clipboard

no data?

Open clcc2019 opened this issue 3 years ago • 4 comments

image image

clcc2019 avatar May 13 '22 01:05 clcc2019

@clcc2019 Did you ever figure out what was going wrong?

benjaminxie avatar Nov 03 '22 21:11 benjaminxie

I ran into this issue as well and I have some suspicions as to what may be going wrong.

Here's some relevant details

  • AKS with containerd as the container runtime
  • Java application with alpine base image (eclipse-temurin:17.0.1_12-jdk-alpine)

Since I'm trying to profile a Java application, the profiler that kubectl-flame will put on my container is async-profiler. According to their docs, I should be seeing output in my application logs similar to the following

--- Execution profile ---
Total samples:           687
Unknown (native):        1 (0.15%)

--- 6790000000 (98.84%) ns, 679 samples
  [ 0] Primes.isPrime
  [ 1] Primes.primesThread
  [ 2] Primes.access$000
  [ 3] Primes$1.run
  [ 4] java.lang.Thread.run

... a lot of output omitted for brevity ...

          ns  percent  samples  top
  ----------  -------  -------  ---
  6790000000   98.84%      679  Primes.isPrime
    40000000    0.58%        4  __do_softirq

... more output omitted ...

Instead, I see something like

WARNING: Kernel symbols are unavailable due to restrictions. Try
  echo 0 > /proc/sys/kernel/kptr_restrict
  echo 1 > /proc/sys/kernel/perf_event_paranoid
perf_event_open failed: Permission denied
perf_event_open failed: Permission denied
perf_event_open failed: Permission denied
perf_event_open failed: Permission denied
perf_event_open failed: Permission denied

If I try either one of echo 0 > /proc/sys/kernel/kptr_restrict or echo 1 > /proc/sys/kernel/perf_event_paranoid, I get sysctl: error setting key '<kernel.kptr_restrict or perf_even_paranoid>': Read-only file system

I think getting my flamegraph means either finding a way to set these two runtime variables or finding a workaround for this.

benjaminxie avatar Nov 04 '22 21:11 benjaminxie

@benjaminxie Does the sysctl command suggested in this comment work?

pvorb avatar Nov 04 '22 23:11 pvorb

@pvorb Not for me :( .

benjaminxie avatar Nov 06 '22 22:11 benjaminxie