OpenMV Entegration Tutorial

[BehicKlncky]

Anyone knows that how to entegrate this engine to OpenMV H7 Plus board ?

[meenchen]

Hi @BehicKlncky, thanks for your interest in our work! To integrate Tinyengine to OpenMV H7 Plus, you will need to modify the firmware of OpenMV and create a Python API for Tinyengine. Please refer to their GitHub repo: https://github.com/openmv/openmv/tree/master/src to build the firmware from the source. If you have further questions, feel free to let me know.

[BehicKlncky]

Hi @meenchen , Actually, integration of your engine to my OpenMV H7 Plus Board was the last step of my whole project, which I decided to start when you released this engine. Now, I've been studying on my project and I'm about the finish which means I came to the last step of it: integration of the engine to the board. So, because of the fact that I should not be considered as a professional on microcontrollers, a little further help would help me a lot. What did you exactly mean by "modifying the firmware" and like how exactly a Python API will I need? If you have an enough time to explain a little detailed for me, I'd be really appreciate it. Thanks a lot for your help.

[senceryucel]

Trying to deal with the same thing. If it is possible, it'd be great to hear from you guys about the integration process of the engine to the OpenMV H7 Plus.

Appreciate your work, thanks a lot in advance!

[meenchen]

Hi @BehicKlncky and @senceryuce, We are still working on it, but I can first give you the general steps.

1. Update Makefile to add TinyEngine source files to SRCS in src/omv/Makefile.
2. Add include path of Tinyengine to OMV_CFLAGS in src/omv/ports/stm32/omv_portconfig.mk(paths to header files of TinyEngine).
3. Add compiled obj paths of Tinyengine to FIRM_OBJ in src/omv/ports/stm32/omv_portconfig.mk. This corresponds to the source files in the 1st step.
4. Remove -Werror flag of CFLAGS in src/omv/ports/stm32/omv_portconfig.mk
5. Add your application implementation code that invokes Tinyengine for inference. Refer to src/omv/modules/examplemodule.c as examples to see how to implement a Python API that calls your C/C++ implementation.

Hope these steps give you a general flow of the integration. If you have a large model, you may also encounter compilation failure due to OOM. In this case, you can disable some OpenMV features that you don't need to free up some space, e.g., those defined in omv/boards/OPENMV4/imlib_config.h.

[meenchen]

Hi @BehicKlncky and @senceryucel,

Recently, we release a tutorial for deploying the VWW demo on the OpenMV cam. Feel free to try it out.

Link of the tutorial: https://github.com/mit-han-lab/tinyengine/tree/master/examples/openmv_vww

[meenchen]

Close due to inactivity. Feel free to reopen.

[senceryucel]

Hello again, @meenchen

@BehicKlncky and I were trying to integrate this beautiful engine into our H7 Plus Board. Everything seemed to be working fine, until we tried to run the example person detection script.

import cexample
import sensor


sensor.reset()  # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565)  # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.HD)  # Set frame size to QVGA 160x128

while True:
    img = sensor.snapshot()  # Take a picture and return the image.
    ret = cexample.person_detection(img, 0.15)

Since we have no LCD screen, we just modified the code a little to test it via frame buffer. However, when we run, it says that there is no module named "cexample".

We looked a bit in the repo for it, but could not find anything that would help us. Any help will be appreciated. Thanks a lot in advance!

[senceryucel]

I decided to open a new issue about it since the problem I am facing with is not directly connected with the problem in the header of this issue.