GraalVM DWARF unwinding not working
Describe the bug
The DWARF unwinder doesn't appear to be working with GraalVM binaries. The framepointer unwinder works so not a big deal at the moment, but we should still understand why and fix it.
To Reproduce
Take the Quarkus example: https://github.com/parca-dev/parca-demo/tree/main/graalvm-quarkus
And remove the -H:+PreserveFramePointer flag from https://github.com/parca-dev/parca-demo/blob/main/graalvm-quarkus/src/main/resources/application.properties
And build with podman build . in the root of the Quarkus example directory, then run it with podman run --rm -it -p8080:8080 <image> on the same host as Parca Agent is running and run a loop of curl http://localhost:8080/hello against it to produce some CPU usage.
Expected behavior
Enabling frame-pointers wouldn't be necessary and the DWARF unwinder would just work.
Screenshots or Profiles (using https://pprof.me)
https://pprof.me/6e9df09
Logs
Verbose BPF logs: https://gist.github.com/brancz/9a6ae5c2eeb78cba1cd661196e603fae
Nothing noticeable in the agent userspace logs, supposedly the unwind tables are even compiled successfully:
level=debug name=parca-agent ts=2023-08-08T15:26:23.308026831Z caller=maps.go:975 component=cpu_profiler component=bpf_maps msg="dealing with main object" mapping="ExecutableMapping {LoadAddr: 0x400000, StartAddr: 0x400000, EndAddr: 0x2da2000, Executable:/usr/bin/quarkus-graal}"
level=debug name=parca-agent ts=2023-08-08T15:26:23.308045085Z caller=maps.go:984 component=cpu_profiler component=bpf_maps msg="adding memory mappings in for executable" executableID=49 buildID=4cc415822512c8cd1f4fdb6a82be16e5170c6f01 executable=/usr/bin/quarkus-graal
level=debug name=parca-agent ts=2023-08-08T15:26:23.308061716Z caller=maps.go:742 component=cpu_profiler component=bpf_maps msg="mapping caching, seen before" buildID=4cc415822512c8cd1f4fdb6a82be16e5170c6f01
level=debug name=parca-agent ts=2023-08-08T15:26:23.308073599Z caller=maps.go:923 component=cpu_profiler component=bpf_maps msg="setUnwindTable called" shards=1 maxshards=50 sumofunwindrows=282542
Software (please complete the following information):
- Parca Agent Version: v0.23.3
- Parca Server Version (if applicable): Polar Signals Cloud
Workload (please complete the following information):
- Runtime (if applicable): quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:22.3-java17
- Compiler (if applicable): Graal
Environment (please complete the following information):
- Linux Distribution: [
cat /etc/*-release] NixOS - Linux Version: [
uname -r] 6.3 - Arch: [
uname -m] amd64 - Kubernetes Version (if applicable): [
kubectl --version] n/a - Container Runtime (if applicable): n/a
As mentioned in #617, it turns out this is likely due to Graal only producing .debug_frame, which we don't have support for (yet).