How do you get executorch to run within Mbed OS?

[ChristophKarlHeck]

Hi guys, We serialized a PyTorch module to a .pte file for Cortex-M architecture by doing this example: https://pytorch.org/executorch/stable/executorch-arm-delegate-tutorial.html). Additionally, we have a P-Nucleo-WB55 development platform. We want to run the module on the development platform using Mbed OS. How do we get the following "torch::executor"-namespace accessible in Mbed OS before we build the binaries that we flash later on the P-Nucleo-WB55? Following is an example of how we would like to do it in Mbed OS:

using namespace torch::executor;
Result<util::FileDataLoader> loader =
      util::FileDataLoader::from("/tmp/model.pte");
assert(loader.ok());

Result<Program> program =
    torch::executor::Program::load(loader.get());
assert(program.ok());

Or is there a better way of integrating the executorch runtime into Mbed OS, or how would you accomplish this task (getting executorch running in Mbed OS on Cortex-M)? Cheers, Christoph

[kimishpatel]

How do we get the following "torch::executor"-namespace accessible in Mbed OS before

Do you mean how to include appropriate headers and link the library? Can you follow the example of executor_runner here https://github.com/pytorch/executorch/tree/main/examples/portable/executor_runner and maybe here https://github.com/pytorch/executorch/tree/main/examples/arm/executor_runner

[kimishpatel]

cc: @iseeyuan

[ChristophKarlHeck]

Thank you for that proposal @kimishpatel. I added the core set of libraries (libexecutorch.a, libportable_kernels.a, libportable_ops_lib.a) and the header files to mbed OS, but I still have dependency issues like

./executorch/extension/data_loader/mmap_data_loader.cpp:16:10: fatal error: sys/mman.h: No such file or directory
   16 | #include <sys/mman.h>

Am I on the right path, or how would you integrate the executor runner?

[mikekgfb]

Thank you for that proposal @kimishpatel. I added the core set of libraries (libexecutorch.a, libportable_kernels.a, libportable_ops_lib.a) and the header files to mbed OS, but I still have dependency issues like

./executorch/extension/data_loader/mmap_data_loader.cpp:16:10: fatal error: sys/mman.h: No such file or directory
   16 | #include <sys/mman.h>

Am I on the right path, or how would you integrate the executor runner?

It seems that the system you're looking at does not have sys/mman.h, the memory management interface of POSIX. What does your vendor suggest to use for memory management when an application requires these interfaces? Or maybe you're missing just an include path and/or build option to use these system services?

IDK whether we have a version that might avoid using memory management - it's just a very fundamental capability that we need to access to load and run the model. (There may be a way for you to emulate these interfaces, but I'm not sufficiently familiar with Mbed OS to suggest a path forward for this system.)

Here's a definition of these functions - you should be able to implement them using your OS, e.g., setting read and write protections. The most "exciting" call is probably mmap which loads a file into your address space with on-demand paging.

However, it would be perfectly acceptable to implement an emulation of the function that fopen()s a file and reads it into memory, with the suitable alignment and all other constraints of the interface definition.

[ChristophKarlHeck]

Hi @mikekgfb, Thank you for your response! I have already solved the problem and published the solution here: https://github.com/ChristophKarlHeck/mbed-torch-fusion-os