Perfetto Chromium Trace Capture should support capturing / exporting to DevTools (bug?)

[ivberg]

Scenario: Devs sometimes use Chromium F12 Dev Tools Perf tab to diagnose performance and see Tasks, or MicroTasks with no additional detail. Perfetto should be the next goto option when needing to dig further into Chromium internals. It is somewhat the next logical goto solution today, but there are gaps, and it's not a superset with at least respect to trace capture (due to a set of bugs?)

Proposal: Perfetto Chromium Trace Capture should support capturing / exporting to DevTools. (BTW I think this is the intention today but it seems partially broken) Result: Devs should be able to easily select a top-level UI option for DevTools which enables devtools categories (devtools.*) and another top-level UI option for JS V8 Stacks (v8.cpu_profiler, v8.runtime_stats_sampling). While Perfetto analysis / timeline might not be able to support / show these, at least the JSON export and import to F12 dev tools should work. Meaning screenshots & JS stacks - along with key browser threads should work in F12 dev tools, while Perfetto shows the extra detail and devs investigating Chromium perf do not have to sacrifice any information between the 2 tools. This flow DOES NOT fully work today.

Context: It seems that the tracing probes and infrastructure are highly shared between F12 perf dev tools & Perfetto. Such that devs cannot capture a F12 Perf trace (with screenshots, JS stacks) AND a Perfetto trace at the same time. So in order to capture all the detail, they need to capture a Perfetto Chromium trace. However, in this flow JS stacks & devtools screenshots are missing. This means they are missing out on key contextual features that were useful (even in export scenario) and while capturing more detailed Chromium internals, they are losing other info & features.

Perfetto can export to JSON and load the JSON as profile in F12 Perf Tab

It seems F12 Dev Tools enables some categories that Perfetto UI Chrome tracing supports. Things like these categories

• v8.runtime_stats_sampling
• v8.cpu_profiler
• devtools.*

[primiano]

+@betasheet who might remember how this all works

[betasheet]

You should be able to enable most of these categories in the Perfetto UI's recording options, i.e. on this page: https://ui.perfetto.dev/#!/record/chrome

The only thing I'm aware of that the Perfetto UI is unable to configure is JS profiling, because it uses V8's profiler rather than Perfetto tracing. Other data like screenshots and trace events should be present in (correctly configured) traces recorded via the Perfetto UI and should convert from there into JSON / be consumable by DevTools correctly.

Are you seeing other missing data? Which data in particular, and are you able to share a proto trace (recorded via Perfetto UI) and a JSON trace (recorded via DevTools) please? :)

[ivberg]

Hey @betasheet both the V8 profiles are missing (even though configured) as well as the screenshots and other devtools features such as timeline. It should be very easy for you to repro this yourself via the details and categories above, and simply compare to what's available in a F12 perf tools trace. I don't think a special repro trace is required. Please do let me know though if you have trouble with the repro.

[LalitMaganti]

Now that https://github.com/google/perfetto/issues/275 is fixed, I don't think there's anything really left to do here.

[ivberg]

This is the opposite of #275. Please read issue in detail. Opposite in that the request here is what the title says "Perfetto Chromium Trace Capture should support capturing / exporting to DevTools". In other words someone could use Perfetto UI to capture a trace that has MORE info than what Chrome Dev tools does, but could export it and hopefully get the stack view that Chromium dev tools has.

Please re-open. I do not have permission

[LalitMaganti]

Ack reopening.

[LalitMaganti]

To be clear though, if I had to invest any time into this issue, I'd prefer to make Perfetto support all the stuff DevTools does rather than improving export. With workspace support, we could easily have a view of the trace inside the Perfetto UI itself which 1:1 matches what's available in DevTools.

[ivberg]

I understand this POV and can understand if it's not a priority. Was documenting this from the other way because devs are used to Chromium, but we may want to capture a more complete Perfetto trace.

[LalitMaganti]

BTW i took another look at this, this is not any problem with export: there is something going wrong at the collection level itself. Specifically, if I enable devtools.screenshots in the UI, I don't see it show up in the proto trace if I convert it to text.

[LalitMaganti]

Apologies, closed by mistake.

I'll escalate this up to the Chrome folk, I don't have enough expertise to debug the Chrome recording interaction. I figured this was a simple export issue but it's actually something deeper wrong.

[LalitMaganti]

Spoke with some Chromium folk. Seems like this is a bug in Chrome tracing itself not in Perfetto. Specifically, https://source.chromium.org/chromium/chromium/src/+/main:content/browser/devtools/protocol/tracing_handler.cc;l=581-588;drc=df9808a2f818e0ad95aabf0de59a31386ba40c1d (which decides which renderer should be traced from) does not get called when tracing outside of DevTools as we do not want to trace a single renderer but all renderes. Unfortunately the screenshot data source only supports a single renderer at a time which makes the whole thing not work.

I believe screenshots is the only thing which should be affected: CPU profiling we know works through Perfetto (as samples show up in the Perfetto UI itself).

@ivberg apart from screenshots, is there anything else missing?

[ivberg]

Screenshots would be nice but not a priority or hard to capture architecture wise. Main ask would be able to take a superset trace with Perfetto primary. And if need be because Perfetto doesn't have stack features to be able to export to a format Dev Tools expects to show the stacks.

If Perfetto has good stack features & comparable to Chromium Dev Tools then not really needed.

[LalitMaganti]

So summarising:

1. Behind the scenes, DevTools is collecting a Perfetto trace as well and then converting that to JSON (this is identical to if you capture a Perfetto trace with the UI and convert using traceconv)
2. Perfetto has always supported slices from DevTools and with recent work now supports V8 stack samples as well
3. Screenshots is the only exception to 1) because of a bug on the screenshots data source

So given all this, it's not clear to me what this bug is tracking beyond screenshots. Is there anything else which is either a) missing in Perfetto UI natively or b) missing in the Perfetto -> Devtools conversion?

[ivberg]

I appreciate the explanation.

This issue is tracking "Perfetto Chromium Trace Capture should support capturing / exporting to DevTools" & "Perfetto can export to JSON and load the JSON as profile in F12 Perf Tab".

Whether or not screenshots is actually part of this is a fine discussion. But a MVP for this issue would be to capture in Perfetto including stacks, and then be able to export in a format F12 Dev Tools could load.

If this is something that won't be supported or worked on than that is ok. I am just explaining what this issue was tracking.

[LalitMaganti]

But a MVP for this issue would be to capture in Perfetto including stacks, and then be able to export in a format F12 Dev Tools could load.

My point is that this should be supported today by doing the following:

1. Collect a trace with the Perfetto UI and download it locally
2. Use traceconv to convert this to json by doing:

./traceconv json <path to proto trace> > <path to json trace>

3. Open this in Devtools.

If something is not working here and is easy to fix, we should fix it. If it's complicated to fix, we will not bother.

[ivberg]

Ah, good to know that this expected to work. My assumption is that it would not work "for free"

Either way, someone should test this. I can test once Perfetto UI is updated to capture the v8 stacks.

I didn't know about traceconv. I assume this is the same in the Perfetto UI option "Convert to .json"?

[LalitMaganti]

I assume this is the same in the Perfetto UI option "Convert to .json"?

Yes.

[LalitMaganti]

Just doing a pass on bugs.

@ivberg given the latest discussion above, is there anything else we want to do here? Canary UI should have the visualisation of V8 stacks so you should be able to check there if it is working correctly for you.

[ivberg]

I tried multiple scenarios with the Canary UI related to above, none of which worked. Do you have steps or a scenario you would expect to work?

1. Per original bug description I enabled these categories in Perfetto UI (Canary) v8.runtime_stats_sampling v8.cpu_profiler devtools.* (I selected all of the devtools options but not sure if this is ideal) a. It would be nice if there were a simple top-level "DevTools Performance" option to replicate what F12 Performance does b. Often there were issues with "start recording" button disappearing (requiring page refresh) b. Often upon refresh UI would navigate away from trace capture to viewing the trace d. Aside from above, a few traces were captured
2. I did not see the DevTools or v8 stacks in Perfetto UI although I may have needed to switch to a special view. There were events in the timeline a. Switching to "Viz" view produced error "Unexpected end of JSON input" even though this should have been a protobuf trace?
3. Used "Convert to JSON" in the UI and attempted to open in F12 Performance. Status:"Recording failed" Description:"No frames found in trace data"

[LalitMaganti]

It would be nice if there were a simple top-level "DevTools Performance" option to replicate what F12 Performance does

Sure but it's quite a bit increase in work for what was requested originally in this bug. It's all possible technically but someone would need to sit down and curate the content.

v8.cpu_profiler

Enabling this category should cause V8 stacks to appear in the UI. I tested this recently so I know it works

Often there were issues with "start recording" button disappearing (requiring page refresh)

This I have to defer to chrome folk, this is handled by the extension which is not owned by our team.

Used "Convert to JSON" in the UI and attempted to open in F12 Performance. Status:"Recording failed" Description:"No frames found in trace data"

This is surprising. Again I tested this and it worked fine

Let me try this and see what is happening.

[LalitMaganti]

did not see the DevTools or v8 stacks in Perfetto UI although I may have needed to switch to a special view. There were events in the timeline

Yes still works:

a. Switching to "Viz" view produced error "Unexpected end of JSON input" even though this should have been a protobuf trace?

Viz page is totally irrelevant here, it's for analysis and was an experiment which went nowhere. We should just hide that page, it's misleading and pretty useless.

Used "Convert to JSON" in the UI and attempted to open in F12 Performance. Status:"Recording failed" Description:"No frames found in trace data"

This also worked just fine for me:

[ivberg]

Some progress in testing

--For Dev Tools--

Used "Convert to JSON" in the UI

Did you convert the trace with the UI or with traceconv? My understanding is that they would produce the same result but perhaps not

Used "Convert to JSON" in the UI and attempted to open in F12 Performance. Status:"Recording failed" Description:"No frames found in trace data"

Still not getting F12 to load with the same error as above

--Regarding testing v8.cpu_profiler-- I tested with just v8.cpu_profiler. I did see null (CPU Stack Samples) under the Renderer and they did show below in the UI when clicked. Selecting a region did show a flamegraph. Yea!!

--Regarding testing settings "matching" DevTools-- I think in my testing with the multiple categories it's easy to fill up the trace buffers fast and lose data. However, I increased trace buffers to max (512) and captured a trace before the buffer filled. a. It hit the same main as the bug reports "Perfetto Chromium Trace Capture should support capturing / exporting to DevTools" b. Since the buffer filled so fast it seems likely these settings are not adequate for "matching" DevTools and we should not only get it to work (issue with trace conversion?) but document at least what settings properly match what DevTools is doing.

[LalitMaganti]

Did you convert the trace with the UI or with traceconv? My understanding is that they would produce the same result but perhaps not

Convert to JSON in UI runs traceconv under the hood so they should be completely identical.

I think in my testing with the multiple categories it's easy to fill up the trace buffers fast and lose data. However, I increased trace buffers to max (512) and captured a trace before the buffer filled.

This was the categories I enabled in the UI for the record:

I also made sure I was actually doing something on the page to make sure there were trace events being generated.

at least what settings properly match what DevTools is doing.

Very unscientific but I collected a devtools trace and then ran it through cat trace.json | jq .traceEvents[].cat | sort | uniq. This produced the following list of categories:

"benchmark,latencyInfo,rail,input.scrolling"
"benchmark,rail"
"cc,benchmark,disabled-by-default-devtools.timeline.frame"
"cc,disabled-by-default-devtools.timeline"
"devtools.timeline"
"devtools.timeline,disabled-by-default-v8.gc"
"disabled-by-default-cpu_profiler"
"disabled-by-default-devtools.screenshot"
"disabled-by-default-devtools.timeline"
"disabled-by-default-devtools.timeline.frame"
"disabled-by-default-v8.compile"
"disabled-by-default-v8.cpu_profiler"
"loading"
"navigation"
"v8"
"v8,devtools.timeline"
"v8.execute"

I imagine using these on the Perfetto UI should give a very close representation of what DevTools is recording.