google
FR: allow including UI macros/startup commands in trace files
It would be amazing to be able to produce Protobuf-based Perfetto trace, in which some of the tracks are actually derived using pre-defined SQL query based on data in another tracks.
E.g., having custom per-thread tracks where track events are tagged (using debug annotations) with associated CPU on which some event happened, it would be nice to have pre-defined per-CPU tracks that combine events across many threads/processes (or whatever custom track scope is) into a "per-CPU" tracks based on debug annotation with CPU number.
Of course, per-CPU is just a one straightforward use case, having full power of SQL at trace emitting stage would be amazingly powerful for producing all sorts of pre-canned views without bloating trace with duplicated data.
@LalitMaganti if I understand correctly, debug tracks can be only created from UI, manually, is that right? The feature request here is basically to be able to produce such debug tracks programmatically inside the trace file itself. But not through duplicating data, but rather deriving debug tracks with SQL, that will come together with protobuf trace file.
This is an intentional design choice we made with Perfetto to separate the visualization of data from the contents of traces. As in, the visualization of traces are independent of how those traces are recorded/written.
This is a learning we took from chrome://tracing which baked the whole viewer itself into the trace but then made it impossible to actually develop due to needing to keep backcompat at the wrong level forever.
As such I don't think we'd want to support something something to the level of "bake SQL queries into the trace". Plugins, the newly added settings and a "initrc" for the UI are IMO far more "Perfetto" ways of dealing with your problem.
@LalitMaganti I understand, thanks for entertaining the idea anyways!
Do you have some links for those added settings and "initrc" for the UI? I'd like to learn about them. Thanks!
Do you have some links for those added settings and "initrc" for the UI?
Settings were added last week so are incredibly new. There's no link I can point you to beyond https://github.com/google/perfetto/commit/b1fc8d6adbb813b7f98b3eabcd6cf7485567a76b
initrc is not actually implemented (as it had a prerequisite on the settings page) but the idea is basically "a bunch of commands which run on startup and are user configurable" similar to a init.rc file in Linux/Android. The basic premise is that you could have a command which is "run all the queries and create debug tracks for the SQL stored in a given setting" and that would be executed on startup, setting up the UI in a user defined way without needing to write a formal plugin and/or bake anything into the file.
@LalitMaganti oh, that's great! Yep, that initrc stuff is exactly what I'd like. Any ETAs for that?
No formal timelines I'm afraid but I think @stevegolton might look at it at some point.
It's definitely on the list. I think we need to work though some unanswered questions first about the syntax, but it's definitely planned.
@LalitMaganti great, thank you, please keep us posted so we can try it when it's available!
How we're planning on implementing this:
- UI macros (just being implemented in https://github.com/google/perfetto/pull/2649): a user defined setting which allows users to specify new "macros" which can be used to compose existing UI commands (including potentially other user defined macros; macros themselves become commands).
- A user defined setting to specify a list of commands to run on UI startup on every trace that user opens
- The ability to specify using URL parameters specific commands which should be run on startup (allowing customization on a per-trace basis, especially powerful once we solve trace sharing)
- (I need to check if this is implemented already or not): the ability to define a debug track from a command
- is now implemented in https://github.com/google/perfetto/pull/2654 and I think once it lands, we can probably close out this issue once you confirm it does what you need.
I verified 4): there is a command to create debug tracks available at https://github.com/google/perfetto/blob/main/ui/src/plugins/dev.perfetto.DebugTracks/index.ts#L29
Update on this since there was a discussion on another bug which I didn't capture here: I think I'm now convinced that we need to have some way to add some mechanism to define within the trace file itself to "run the following commands on UI startup": it will tie to the same automation commands feature that works through settings.
See more discussion in https://github.com/google/perfetto/pull/2654