embassy
embassy copied to clipboard
embassy-rs
InterruptExecutor doesn't work in rp2040 multicore.
InterruptExecutor works by storing the interrupt number in the executor's Pender. So when a task is woken, the executor's interrupt number is pended in the NVIC. This causes the executor to get polled, which will poll the task.
The issue is if we have 2 cores, we actually have 2 NVICs. So:
- The executor is started on core0
- The executor's irq is enabled in core0's NVIC
- a waker gets sent from core0 to core1
- the waker is woken from core1
- the irq gets pended in core1's NVIC
- which does nothing because it's not enabled in core1's NVIC, only core0's.
The fix is:
- embassy-executor should get a "assume single core" Cargo feature or something, so it can choose to not expose
InterruptExecutorin multicore. - embassy-rp could expose a RP2040-specific InterruptExecutor, with some extra care to store "core num + irq num" in Pender, and make cross-core interrupts work. Maybe through FIFO?
Ideally it should also be possible to use the same irq number in the 2 cores simultaneously, for different InterruptExecutor instances. This requires the different cores to have different vector tables though, which cortex-m-rt doesn't support.
Could we encode the desired nvic(s) in the irq number, and then dispatch somehow when pending? there's only 32 irqs, 64 for both nvic, so plenty of bits left. :)
(-: Feel free to ignore this comment :-)
The fix is:
* embassy-executor should get a "assume single core" Cargo feature or something, so it can choose to not expose `InterruptExecutor` in multicore. * embassy-rp could expose a RP2040-specific InterruptExecutor, with some extra care to store "core num + irq num" in Pender, and make cross-core interrupts work. Maybe through FIFO?
Having multi core support in embassy-rp will restrict embassy multi core support to RP2040 devices, will hinder embassy supporting other multi core devices.
Giving embassy-executor feature like core count seems to me the better long term option.
Could we encode the desired nvic(s) in the irq number, and then dispatch somehow when pending?
That sounds like only ARM has multi core devices. I do like to see that Embassy could support ESP32 (dual core) and multi core RISC-V devices.
Having multi core support in embassy-rp will restrict embassy multi core support to RP2040 devices, will hinder embassy supporting other multi core devices.
No, this doesn't prevent anyone else from writing multicore support for other chips in other chip-specific crates.
It is actually impossible to add "ARM multicore support" to embassy-executor. ARM doesn't define how multi-core communication works, each vendor invents their own IPC/FIFO peripherals. So to write a multicore executor you must use vendor-specific stuff. It can't go in embassy-executor.
On Sun, Aug 13, 2023 at 12:42:36PM -0700, Dario Nieuwenhuis wrote:
Having multi core support in embassy-rp will restrict embassy multi core support to RP2040 devices, will hinder embassy supporting other multi core devices.
No, this doesn't prevent anyone else from writing multicore support for other chips in other chip-specific crates.
It is actually impossible to add "ARM multicore support" to
embassy-executor. ARM doesn't define how multi-core communication works, each vendor invents their own IPC/FIFO peripherals. So to write a multicore executor you must use vendor-specific stuff. It can't go inembassy-executor.
OK, sounds good. Thanks for sharing.
Groeten Geert Stappers
Silence is hard to parse
It is actually impossible to add "ARM multicore support" to embassy-executor
At this point I believe the executors in embassy-executor should explicitly be declared single-core only.
- ESP32 Xtensa needs peripheral trickery to pend an interrupt across cores. ESP32's
waitiinstruction is also not aware of other cores' interrupt requests. - I don't know anything about RISC-V
- As per this issue, cortex-m seems to have the same issue as Xtensa
- I discount std and wasm
In ESP-land we have 4 specific interrupts, implemented outside the CPU cores, that are software pendable from either core, and can be handled by both cores. From skimming the datasheet it looks like the PIO could be used to emulate similar functionality on the RP without touching the FIFOs inside.
It should also be noted that executors can be implemented outside embassy-executor without disadvantage. It's also not very difficult to build an executor on top of raw::Executor, and embassy-executor could even provide building blocks to make chip-specific implementations minimal. This makes me wonder if embassy-executor actually needs to provide architecture-specific implementations.
