ipc4: implement mtrace logging support
Set of patches to implement mtrace support in SOF firmware.
Requires Zephyr and Zephyr PR https://github.com/zephyrproject-rtos/zephyr/pull/49147 to work.
V1 update:
- otherwise ready to, but some hacks in this version related to sending notifies when a IPC message is under processing -- a crude hack is this patch to solve the issue, but we probably need something a bit cleaner
For the record, I'm looking for something hard coded and simple to get data out as the first merge point. We can then improve with dynamic config/tuning in subsequent updates.
V3 update:
- major clean up, this should be now ready for @lgirdwood 's criteria of a minimal working set
- keeping still as draft as the Zephyr PR needs to be merged first. I can then undraft this and submit this together with a manifest update
- the aging timer task is a bit ugly still, but I had major issues with other options I tried - trying to use schedule_task from logging hook results in recursion and non-functional FW - k_timer and k_work deliver callbacks in irq context (and possibly on non-zero core) and I can't figure out how to send the IPC notifies safely from this context
V4 update:
- implemented the worker thread using native interfaces -- a bit more code, but this is more efficient as we don't need to run anything extra in the 1ms ll cadence
- added overlay changes to make this the default for all IPC4 platforms
- fixed a few bugs w.r.t. draining logs at end when streaming is stopped
- still keeping as draft as the Zephyr side PR is still in review
All, the important thing here is that we need some data out as the first step. Today we have no logging. We can improve our logging infra incrementally (and we can track this with FEATURES).
V5 update:
- improved the SOF EDF adaptation to Zephyr a bit and implemented the mtrace work thread logic using edf_schedule.h (this is basicly a k_thread pinned on a thread... and it's a cost we are already taking so why not piggyback on this)
- bugfix to handle runtime suspend cases to mtrace-reader.py
- still keeping as draft as the Zephyr part is not merged
V6 update:
- not included a manifest update to bring in the Zephyr dependencies
- a few bugfixes to the EDF adaptation for recurring EDF tasks
- added a more clear TODO for multicore; I spent a lot of time trying to use notifier.h to wake up a task from one core to another, but kept hitting with weird issues... this version now explicitly optimized for single-core usege. it will work wiht multicore, but one may need to use a more aggressive aging timer not to lose logs
Zephyr builds errors are due to old SDK used in CI. 0.15 SDK is needed to build (or XCC).
@keqiaozhang @fredoh9 @marc-hb can we update the Zephyr SDK to align with the version used by Zephyr. Thanks
V7 update:
- updated github workflows to use a specific zephyr docker-image that containts 0.15 SDK
- there is one IPC4 build failure left but that is a compiler warning handled via https://github.com/thesofproject/sof/pull/6211
- no change to code otherwise
@keqiaozhang @fredoh9 @marc-hb can we update the Zephyr SDK to align with the version used by Zephyr. Thanks
Pls ignore, @kv2019i has updated in the PR.
V8 update:
- fix XTOS IPC4 build caught by CI job "checktree (IPC_VERSION=IPC4 UNSIGNED_list= SIGNED_list='tgl tgl-h cnl icl jsl')"
V9 update:
- addressed @abonislawski 's revies and most notably reviewed mtrace.[ch] to logging.[ch] as the same IPC can be used for other logging backends
- one bugfix to disabling logs that was causing IPC timeouts in testing
- kept the SDK update... the "latest" tag still points to 0.14 SDK
Zephyr IPC4 build fix merged.