depthai
depthai copied to clipboard
Extend Image Manip Node Capabilities
Start with the why:
Now that DepthAI users are building increasingly complex pipelines, we are finding that the Image Manip node is being needed to do functions that are not yet supported.
Specifically, it's going to be quite important to have ImageManip being able to take NV12 and produce NV12 as well, to be able to later encode it into either MJPEG or H26x as on the POE models, the effective bandwidth is smaller than USB2. And for example, April Tags ( https://github.com/luxonis/depthai-python/pull/298 ) and likely other common pipelines will need conversion from RGB to grayscale. And for PoE and SPI if is often quite useful (and or strictly necessary) to be able to do MJPEG/H26x after various pipelines, so having both NV12->RGB and RGB->NV12 will be quite valuable for such cases.
Move to the how:
Take advantage of the underlying hardware-accelerated image manipulation functions and integrate these into the Image Manip node.
Move to the what:
Improve ImageManip to be able to
- [ ] Take in NV12 and produce NV12
- [ ] NV12->RGB and RGB->NV12
- [ ] Support RGB to gray conversion
- [ ] Support upscaling
- [ ] Support Distortion Correction and Rectification (which is currently only in the stereo node).
- [ ] Support a wide variety of color space conversions in Image Manip Node
- [ ] Improve
setKeepAspectRatio(False)behavior as per here
A combination of distortion correction, crop and concatenation would be really nice for simple cases of image stitching.