[Feature] Select user data size for each hub

[laurensvalk]

Programs can now be saved on all hubs. This issue is to define exactly how much space we'll make available on each hub.

To make a more informed choice, here is the current state of affairs for all hubs.

All sizes in KB	Move Hub	City Hub	Technic Hub	Essential Hub	Prime / Inventor Hub
Flash	128	256	256	1024	1024
External raw flash	-	-	-	1023	1023
External file system	-	-	-	3072	31744
Bootloader	20	20	32	32	32
RAM	16	32	64	320	320
FW @ c126dd	99.6	167.1	177.1	220.7	226.3
BSS @ c126dd	4.6	4.2	6.9	18.4	23.0
Stack (1)	~~2~~ 3	4	8	16	16
Remaining RAM (2)	~~9.4~~ 8.4	23.8	49.1	285.6	281.0
Max storage (3)	8.4	68.9	47.0	1023	1023
Proposed storage (4)	4	16	16	256	256

---

1. stack is also a design choice, also up for grabs.
2. Calculated as: Remaining RAM = Total - BSS - Stack. Technically it is a bit less due to other small RAM sections.
3. Calculated as: max(Flash size - bootloader size - FW size, external raw)
4. This should be less than min(remaining RAM, max storage). If we want to never decrease this, we should leave some wiggle room, as proposed here.

---

Previously, user program size was implicitly limited to 25% of RAM (half for heap, then half again to be able to copy large MPY files). So, on all hubs this is either kept the same or improved (and by some margin).

[laurensvalk]

Stack is also a design choice, also up for grabs. Thoughts welcome.

4k is probably a bit large for the move hub. We could try 2k and see how many people actually get a stack overflow.

Originally posted by @dlech in https://github.com/pybricks/pybricks-micropython/pull/106#discussion_r940598127

With 1K for exception handling, this leaves just 1K for the MicroPython stack, which isn't much. This program trips it already (needs 1116):

from micropython import mem_info

def func12():
    mem_info()

def func11():
    func12()

def func10():
    func11()

def func9():
    func10()

func10()

We can probably set this to 3K or more, so at least 2K for MicroPython. Since RAM wasn't the constraining factor on this hub, it doesn't reduce the allowed program size anyway.

[laurensvalk]

This is a bit more work than just changing the values, because the bootloader will be checking a portion of the data.

[dlech]

from pybricks import version
from micropython import stack_use

def f(depth):
    print(depth, stack_use())
    f(depth + 1)

print(version)
print("top", stack_use())
f(1)

outputs (on older firmware):

('movehub', '3.2.0b3', 'v3.2.0b3-10-g6acc817d-dirty on 2022-08-01')
top 436
1 668
2 900
3 1132
4 1364
5 1596
6 1828
7 2060
8 2292
9 2524
10 2756
11 2988
Traceback (most recent call last):
  File "main.py", in <module>
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
  File "main.py", in f
RuntimeError: maximum recursion depth exceeded

This shows us that MicroPython requires 436 bytes of stack to just run and 232 bytes each for a simple function call.

So with a 1K safety factor, a 2K stack would get us two levels deep, a 3K stack gets us 6 levels deep and a 4K stack gets us 11 levels deep.

[dlech]

We could also consider backing down the 1K safety factor but we would need to be quite certain that library (pbio+3rd party) calls could never exceed that amount.

As an experiment, we could record the minimum sp register value in the systick interrupt, then let the hub run for a while running motors, using Bluetooth, etc to get an idea of how much stack is actually used. Of course this relies on random chance that the interrupt occurs during max stack usage, so it wouldn't be a 100% guarantee that we would get an exact value, but it could get us in the ballpark.

[laurensvalk]

This is all done. Let's reopen if we want to change these values.

[laurensvalk]

The values were changed in https://github.com/pybricks/pybricks-micropython/commit/2e97a74e759f876ab9fb52603fb0bc687f413bb6.

[BertLindeman]

To help TechnicHub users to save the exception in their program after a connect to an xBox controller, I try to write the exception to user storage so it can be shown later. Maybe a bit too advanced and not really handleable, but I want to try.

Is there a variable already available in beta version(s) to indicate: "for this hub in this firmware level" the userdata starts "here"?

Would that be including gyro settings etc?

[laurensvalk]

The user data offset always starts at 0, so there is no issue writing there. You cannot write outside of the user area.