`perfcollect install` fails on Ubuntu Bionic containers with certain kernels

[mjrousos]

Running ./perfcollect install in an Ubuntu container run on some non-Ubuntu hosts (such as in an Azure AKS cluster) fails because Ubuntu perf packages are specific to the OS's kernel version and no perf package is available for some of these configurations.

One example would be to run a container based on mcr.microsoft.com/dotnet/core/sdk:2.2-bionic in an Azure AKS cluster. Running percollect install on such a container fails to install perf with this error:

root@perf-sample-bionic-7845648885-n8zpc:/app# ./perfcollect install
Installing perf_event packages.
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package linux-tools-4.15.0-1055-azure
E: Couldn't find any package by glob 'linux-tools-4.15.0-1055-azure'
E: Couldn't find any package by regex 'linux-tools-4.15.0-1055-azure'
E: Unable to locate package linux-cloud-tools-4.15.0-1055-azure
E: Couldn't find any package by glob 'linux-cloud-tools-4.15.0-1055-azure'
E: Couldn't find any package by regex 'linux-cloud-tools-4.15.0-1055-azure'

The issue is that perfcollect attempts to install a perf package based on the kernel version (4.15.0-1055-azure for my AKS hosts) but no perf packages exist for that kernel.

I've tried installing a similar perf package (linux-tools-4.15.0-1037-azure) as a workaround but ran into the following error while stopping trace collection (the resulting trace had no CPU stacks): ./perfcollect: line 649: 22605 Segmentation fault (core dumped) $* >> $logFile 2>&1

A workaround that does appear to work is to clone the Linux kernel repository and build tools/perf locally. Using that version of perf allowed perfcollect to run successfully, but it's a fairly inconvenient workaround.

[mjrousos]

I've found another workaround that's a bit simpler than building Perf from source but still not ideal.

Although there aren't Perf packages available for Bionic on recent Azure Linux kernels, it looks like that are Perf packages available for recent Azure Linux kernels on Xenial!

I tried copying a version of Perf from a Xenial package for the correct kernel version to a Bionic container in AKS and it worked great.

Perhaps we can figure out how these Xenial packages are being published and have them made available for Bionic, too? Or, barring that, restore Xenial versions of Perf if Bionic ones aren't available when installing PerfCollect?

[tdihp]

Found the same situation when trying mcr.microsoft.com/dotnet/core/samples:aspnetapp with AKS. The perfcollect can be installed on node without issue, however when collecting from node directly with pid, I'm not able to get useful information.

[nmoreaud]

~/projects/cnx-experimental$ sudo ./perfcollect install Installing perf_event packages. Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package linux-tools-5.4.72-microsoft-standard-WSL2 E: Couldn't find any package by glob 'linux-tools-5.4.72-microsoft-standard-WSL2' E: Couldn't find any package by regex 'linux-tools-5.4.72-microsoft-standard-WSL2' E: Unable to locate package linux-cloud-tools-5.4.72-microsoft-standard-WSL2 E: Couldn't find any package by glob 'linux-cloud-tools-5.4.72-microsoft-standard-WSL2' E: Couldn't find any package by regex 'linux-cloud-tools-5.4.72-microsoft-standard-WSL2'

[brianrob]

Generally, this issue should now be solved with #1278.

@nmoreaud, it looks like you're trying to run this on WSL2, and so there just may not be packages that match this format for WSL2. I have this on my list to investigate, but haven't gotten a chance yet. The first step would be to figure out if there are packages for WSL2 to install perf. If you're interested in starting the investigation, that would be great, but no worries either way.

[nmoreaud]

I searched about it yesterday, perf tool seems to have gained support recently on WSL 2 kernel. However, people who managed to use it compiled it from sources. I won't have the time to investigate this.

My aim is to profile C# .NET5 code on WSL2/Ubuntu with VSCode, I though there would be lots of tutorials but VSCode seems to lack profiler integration.

[nmoreaud]

Maybe we should have a warning about WSL support in perfcollect documentation