wokwi-features icon indicating copy to clipboard operation
wokwi-features copied to clipboard

Improve support for Zephyr RTOS applications

Open kartben opened this issue 2 years ago • 62 comments
trafficstars

The zephyr-esp32 builder is already a great start in allowing to run Zephyr apps from Wokwi. It would be great to look into generalizing/improving support by allowing the users to configure things such as:

  • providing a prj.conf
  • providing DT overlays
  • supporting more than just basic esp32, but also potentially rp2040, esp32-s2, etc.

I am around to chat more around this, and I will provide help as much as I can!

kartben avatar Feb 08 '23 14:02 kartben

Hi Benjamin! Thanks for offering your help here. Zephyr apps is definitely something I'd love to see running in Wokwi.

Right now, we have three ways to use Wokwi:

  1. Online, though the web interface - very popular, but doesn't like long build times or complex build environments
  2. As a Visual Studio Code extension - in public beta, and you're already familiar with it
  3. As a command-line tool for CI use cases - in private beta, but we plan to open it publicly later this year

Which use cases do you have in mind?

urish avatar Feb 08 '23 20:02 urish

I would love to make sure #1 gets improved, at least to the point where it allows people to get a better understanding of how they can tinker with things like device tree overlays, and project configuration. Point taken though regarding long build time possibly being a problem. I need to play more with the VS Code extension to get a sense of how the current experience is. It might be that it just works perfectly already :-)

Cheers!

kartben avatar Feb 12 '23 14:02 kartben

How complex is it to set up a Zephyr ESP32 "Hello world" project in VS Code?

urish avatar Feb 12 '23 19:02 urish

How complex is it to set up a Zephyr ESP32 "Hello world" project in VS Code?

Not too complex, apparently :)

image

kartben avatar Feb 13 '23 18:02 kartben

Damn, you are fast!

urish avatar Feb 13 '23 18:02 urish

Is there a repo with this setup?

urish avatar Mar 23 '23 22:03 urish

@kartben heads up on wokwi-cli, which will allow running any project you set up for VS Code in CI. Want to give it a go?

urish avatar May 04 '23 20:05 urish

@kartben heads up on wokwi-cli, which will allow running any project you set up for VS Code in CI. Want to give it a go?

Yes, please! Is everything I'd need available at the linked URL? And I need to do a proper write-up of some of my experiments to date with the VS Code extension.

kartben avatar May 04 '23 20:05 kartben

Yes, please! Is everything I'd need available at the linked URL?

It should - but this is pretty early, so you'll probably find something I forgot. Ping me when you do.

And I need to do a proper write-up of some of my experiments to date with the VS Code extension.

Yes, please!

urish avatar May 04 '23 20:05 urish

wokwi-cli is really easy to use. I got it running here https://github.com/bmeisels/wokwi-cli-github-actions-example and @urish even added a dedicated action https://github.com/wokwi/wokwi-ci-action.

bmeisels avatar May 06 '23 21:05 bmeisels

@kartben which app is this? can you please share the repo?

image

urish avatar Aug 04 '23 13:08 urish

@kartben which app is this? can you please share the repo?

image

I don't remember 😭 I really need to put together a quick README somewhere...

kartben avatar Aug 12 '23 17:08 kartben

People keep asking about Zephyr + Wokwi. We need your README :)

urish avatar Aug 14 '23 21:08 urish

@kartben which app is this? can you please share the repo?

image

@urish ok so I am finally spending some time digging this up. Here's a working, self-contained, project: https://github.com/kartben/wokwi-zephyr-projects/tree/master/esp32c3-lvgl-with-mpu6050

Things are unfortunately really slow, and it's not clear why. I vaguely remember discussions about performance issues depending on how sleep is implemented, and also the ILI9341 simulation potentially being slow. I will try a similar sample with the RP2040, just in case I have better luck. In the mean time, your input on how to potentially improve the performance would be most welcome!

kartben avatar Oct 03 '23 14:10 kartben

Oh yeah!

image

In general, transferring huge amount of data over SPI is slow. There's an experimental way to short-circuit the low level SPI stuff and speed things up. Also, DMA can help to some extent - do you know of the underlying code uses DMA?

urish avatar Oct 03 '23 14:10 urish

Oh yeah!

w00t!

In general, transferring huge amount of data over SPI is slow. There's an experimental way to short-circuit the low level SPI stuff and speed things up. Also, DMA can help to some extent - do you know of the underlying code uses DMA?

Ya but I think it's more than just SPI, let me try to produce a minimal repro sample

kartben avatar Oct 03 '23 14:10 kartben

Meanwhile, I ran some benchmarks using the Wokwi Profiler. Seems like most of the ESP32 time is spent memcpy'ing and doing SPI stuff:

image

urish avatar Oct 03 '23 14:10 urish

Nice (the profiler I mean, not the fact that LVGL is probably a huge performance killer :D). I can't easily run it myself tho, right? From the Web UI it tries to run from source, and I don't think I can launch it from VS Code?

Please see https://github.com/kartben/wokwi-zephyr-projects/tree/master/esp32c3-blinky It should blink an LED every 1sec, but looks like the timing is off?

...
load:0x403d0000,len:0x2370
entry 0x403ce000
W (280) rtc_init: o_code calibration fail

*** Booting Zephyr OS build v3.5.0-rc1-11-g175560ff7e50 ***
LED state: ON @ 0
LED state: OFF @ 2129
LED state: ON @ 4258
LED state: OFF @ 6387

Could the rtc_init warning be relevant?

kartben avatar Oct 03 '23 15:10 kartben

currently testing with esp32s3 and the timing for blinky seems correct

kartben avatar Oct 03 '23 15:10 kartben

Thanks! Yeah, the profiler is not available (yet) in VS Code, but I'm working on it!

Please see https://github.com/kartben/wokwi-zephyr-projects/tree/master/esp32c3-blinky

Good catch, pushed a fix. Can you please test again?

urish avatar Oct 03 '23 19:10 urish

Please see https://github.com/kartben/wokwi-zephyr-projects/tree/master/esp32c3-blinky

Good catch, pushed a fix. Can you please test again?

Yay! Much better :) it looks like it speed things up a tiny bit for the lvgl sample, but that might be magical thinking. I can't seem to get SPI to work for ESP32S2 and S3 (pretty sure I'm just doing something wrong on the Zephyr side) but before I invest more time in trying to debug this, is there a chance that Xtensa simulation will be faster than RISC-V's?

kartben avatar Oct 03 '23 19:10 kartben

Great!

It's hard to tell whether Xtensa is going to be faster - it depends on the compiler and esp-idf. For instance, it might use DMA on the S2 (or original ESP32), and that could considerably speed things up.

When you pause the simulation with S2/S3, do you see the SPI pins configured correctly?

image

urish avatar Oct 03 '23 20:10 urish

Another update: got rid of the "rtc_init: o_code calibration fail" issue, so now the program should start much faster for C3 and S3 (it won't keep polling the RTC until it time outs).

urish avatar Oct 03 '23 20:10 urish

Another update: got rid of the "rtc_init: o_code calibration fail" issue, so now the program should start much faster for C3 and S3 (it won't keep polling the RTC until it time outs).

very nice -- much faster indeed!

kartben avatar Oct 04 '23 08:10 kartben

When you pause the simulation with S2/S3, do you see the SPI pins configured correctly?

ooh, what a neat feature :) So I am pretty sure things are wired up properly, and it looks like the pins are also OK, see below. I've put the non-working sample here https://github.com/kartben/wokwi-zephyr-projects/tree/master/esp32s3-lvgl-with-mpu6050, in case you can maybe spot anything obvious when tracing. Note that Zephyr+I2C+MPU6050 seems to work just fine for this same diagram, it's SPI that seems busted. Thanks!!

image

kartben avatar Oct 04 '23 08:10 kartben

Thanks for making it easy to reproduce - I found out it was a bug in the simulator (SPI hardware was not signaling the code it finished transferring data correctly). Uploaded a fix - I can see it now draws on the screen, but then the program (running in the sim) seems to crash with a exception right after it finishes drawing. Is that expected?

Reading symbols from ./esp32s3-lvgl-with-mpu6050/zephyr.elf...
Remote debugging using localhost:3333
warning: multi-threaded target stopped without sending a thread-id, using first non-exited thread
_DoubleExceptionVector () at /Users/kartben/zephyrproject/zephyr/arch/xtensa/core/xtensa-asm2-util.S:444
444     /Users/kartben/zephyrproject/zephyr/arch/xtensa/core/xtensa-asm2-util.S: No such file or directory.
(gdb) bt
#0  _DoubleExceptionVector () at /Users/kartben/zephyrproject/zephyr/arch/xtensa/core/xtensa-asm2-util.S:444
#1  0xffffffff in ?? ()

urish avatar Oct 04 '23 11:10 urish

Thanks for making it easy to reproduce - I found out it was a bug in the simulator (SPI hardware was not signaling the code it finished transferring data correctly). Uploaded a fix - I can see it now draws on the screen, [...]

cool, works for me too! Thanks!

but then the program (running in the sim) seems to crash with a exception right after it finishes drawing. Is that expected?

Reading symbols from ./esp32s3-lvgl-with-mpu6050/zephyr.elf...
Remote debugging using localhost:3333
warning: multi-threaded target stopped without sending a thread-id, using first non-exited thread
_DoubleExceptionVector () at /Users/kartben/zephyrproject/zephyr/arch/xtensa/core/xtensa-asm2-util.S:444
444     /Users/kartben/zephyrproject/zephyr/arch/xtensa/core/xtensa-asm2-util.S: No such file or directory.
(gdb) bt
#0  _DoubleExceptionVector () at /Users/kartben/zephyrproject/zephyr/arch/xtensa/core/xtensa-asm2-util.S:444
#1  0xffffffff in ?? ()

mmmh no (granted I don't have actual hardware to test) :) here's the line if that helps https://github.com/zephyrproject-rtos/zephyr/blob/ecefcd7f87d73bbd7d472399127421639d5bc2c2/arch/xtensa/core/xtensa-asm2-util.S#L444

On my side I can't seem to get it to crash and it rather looks like it's stuck? Simulation rate drops to ~15%, and I can't seem to get a crashdump?

kartben avatar Oct 04 '23 11:10 kartben

Yeah, same here - it it stuck in the _DoubleExceptionVector handler. Not sure why simulation speed drops to 15% in this case, but the question is what happened just before it got to _DoubleExceptionVector.

urish avatar Oct 04 '23 11:10 urish

Yeah, same here - it it stuck in the _DoubleExceptionVector handler. Not sure why simulation speed drops to 15% in this case, but the question is what happened just before it got to _DoubleExceptionVector.

This time the problem was on my side! Increased stack size of the app and now it works! 🥳 I've updated the binaries accordingly.

kartben avatar Oct 04 '23 12:10 kartben

Awesome!

Meanwhile, I also pushed a prototype of short-circuit the SPI to slightly speeds things up. You can enable it by setting the "__labs_spiAccel" attr to "1", e.g.

    {
      "type": "board-esp32-s3-devkitc-1",
      "id": "esp",
      "top": -28.98,
      "left": -120.23,
      "attrs": {"__labs_spiAccel": "1"}
    },

urish avatar Oct 04 '23 12:10 urish