profiler icon indicating copy to clipboard operation
profiler copied to clipboard

Avoid the need for `perf script` when importing from perf

Open larseggert opened this issue 1 year ago • 6 comments

It's a bit annoying that one needs to convert a perf trace with perf script before it can be loaded into the profiler, because that explodes the file size by an order of magnitude.

┆Issue is synchronized with this Jira Task

larseggert avatar May 06 '24 13:05 larseggert

Ah yes, we should update our documentation to point to samply instead: Use samply import perf.data and it will open the Firefox profiler with the converted profile, and overall be much faster than running perf script first.

Dragging the perf.data file itself into the Firefox Profiler isn't really something we can make work; the perf.data file doesn't carry any symbol information. The symbols come from other files on the file system.

mstange avatar May 06 '24 20:05 mstange

I'm not sure if samply works for us, since we run perf in CI and then export the data as CI artifacts, for download and analysis on local machines.

larseggert avatar May 07 '24 06:05 larseggert

Cool, can you give me a link to the script that runs perf as part of your CI? I'm curious.

If the symbols are only on the CI machine, then the symbolication has to happen on the CI machine. At the moment samply does not have a way to put the symbols into the generated profile.json; the symbols are added once the profile has been loaded by the front-end, with separate symbolication requests to the local samply server.

If the symbols are also uploaded on some kind of symbol server, then theoretically you could generate unsymbolicated profile.json files in CI (using samply import perf.data -o profile.json --save-only and symbolicate them on demand when developers look at the collected files. But this would not be a very ergonomic solution.

So I think for now you'll need to keep using perf script until I implement upfront symbolication in samply.

mstange avatar May 07 '24 20:05 mstange

Cool, can you give me a link to the script that runs perf as part of your CI? I'm curious.

https://github.com/mozilla/neqo/blob/beffcc6ca9b19bcf605aa3d36927da887c7eb6f4/.github/workflows/bench.yml#L173-L183

larseggert avatar May 08 '24 06:05 larseggert

Very cool! In case others are curious, here's an example profile I got from one of the zip files on one of the PRs: https://share.firefox.dev/3wvM5uR

mstange avatar May 09 '24 20:05 mstange

Chiming in that it would be nice if there was a CLI way to go directly from perf.data into a profile with symbolication that could be exported and viewed in firefox profiler. We also want to generate perf data in CI, and viewing the flamegraphs in samply / firefox profiler is much nicer when there is a lot of multithreading (its a rust binary heavily using Rayon). Ther perf script output is over 2GB so I believe the firefox profiler isn't going to allow uploading it from file.

Also open to directly using samply without perf, but AFAIK there's no way for samply to store symbols in an exportable format.

jonathanpwang avatar Feb 05 '25 18:02 jonathanpwang