Get MobileStereoNet Running on DepthAI

[Luxonis-Brandon]

trafficstars

Start with the why:

Algorithmic stereo depth is useful because it's fast and relatively inexpensive (computational-budget, power-budget, and latency-budget). However, it is easily fooled in specific scenes. And these cases can be painful, requiring the addition of illumination that adds information to the scene (such as laser dot projectors).

ML-based depth brings with it all the advantages of ML approaches. It can learn ways to not be fooled if provided with enough training data. And with the advent of nice 3D training datasets, and synthetic generation, this is increasingly useful.

However, such approaches are often extremely computationally heavy, making them high-enough computational complexity that they are not realistically possible in many edge- and perhaps all embedded- use-cases.

The newly-released mobilestereonet (here) may change that. Given that it's MobileNetV1 or MobileNetV2 based, there's a decent chance it may be low-enough compute to run performantly on DepthAI. For example MobileNetV2 Deeplabv3+ can run at at least 27FPS on DepthAI, and likely over 30FPS with some coming firmware improvements/optimizations.

Move to the how:

Given this is MobileNetV1 or MobileNetV2 based, it may be relatively straightforward to get running on DepthAI, just like DeeplabV3+ was.

Move to the what:

Get the Neural-based stereo matching network MobileStereoNet running on DepthAI.