PNG support for the SmartCam's image classifier

[georgeslabreche]

Here are some pointers on how to get started with png support for the SmartCam's image classifier.

A. Getting familiar with the development environment/setup

A1. Go through this readme that provides some context on how the image classifier's source code is setup:

A2. Try to compile it on your local machine:

Note how there are two compilation options: one for the default GNU toolchain (assuming you're on Linux) and the other for the ARM32 environment (destined for SEPP). This is important for later (details in A3).

Don't forget to run the cleanup script.

A3. In our case, we are interested in updating the image_classifier executable binary to support png. When that's done, we will simply replace the binary and shared objects that are located in the SmartCam's repo. Specifically here for ARM32

And here for the local Linux environment that we run on development machines for testing purposes.

B. Implementing support for png

B1. We need to replace the dependency with ujpeg.cc/ujpeg.h, located here.

Which is used to read/decode the jpg image input and load the RGB pixel values into memory in an array.

Note how the 0-255 pixel values are rescaled given the input mean and standard deviation, we will need to make sure that this logic is preserved after ditching/replacing ujpeg.

B2. Use the stb library instead of ujpeg, found here.

In particular the stb_image image loader header.

B3. The documentation in the stb_image.h file is great but we can also consult a straightforward example implementation with this image_resizer that was written for SmartCam which also uses stb.

In particular, how to include the stb_image header.

And [how to load the image file data into an RGB data array] (https://github.com/georgeslabreche/image-resizer/blob/e4698219353a37b4b49e15935180c202e8f9aeba/src/main.c#L106-L107).

Note how it is achieved in one line of code with any image type when using stb versus the 20+ with ujpeg.

B4. And that's it, recompile as instructed in A2 and redeploy as instructed in A3. Please use feature branching.

We will have to update the BUILD file to remove configurations that include the ujpeg library and replace them with the new dependency on stb_image.

Here, here, and here.

Also, don't forget to delete the ujpeg.h and ujpeg.cc files.

C. Versioning

Let's make sure to use release tagging and versioning on both repos. For the image classifier: https://github.com/georgeslabreche/tensorflow-opssat-smartcam/releases

And for the SmartCam with the updated image classifier binaries: https://github.com/georgeslabreche/opssat-smartcam/releases