depthai icon indicating copy to clipboard operation
depthai copied to clipboard

Extend Image Manip Node Capabilities

Open Luxonis-Brandon opened this issue 4 years ago • 1 comments

Start with the why:

Now that DepthAI users are building increasingly complex pipelines, we are finding that the Image Manip node is being needed to do functions that are not yet supported.

Specifically, it's going to be quite important to have ImageManip being able to take NV12 and produce NV12 as well, to be able to later encode it into either MJPEG or H26x as on the POE models, the effective bandwidth is smaller than USB2. And for example, April Tags ( https://github.com/luxonis/depthai-python/pull/298 ) and likely other common pipelines will need conversion from RGB to grayscale. And for PoE and SPI if is often quite useful (and or strictly necessary) to be able to do MJPEG/H26x after various pipelines, so having both NV12->RGB and RGB->NV12 will be quite valuable for such cases.

Move to the how:

Take advantage of the underlying hardware-accelerated image manipulation functions and integrate these into the Image Manip node.

Move to the what:

Improve ImageManip to be able to

  • [ ] Take in NV12 and produce NV12
  • [ ] NV12->RGB and RGB->NV12
  • [ ] Support RGB to gray conversion
  • [ ] Support upscaling
  • [ ] Support Distortion Correction and Rectification (which is currently only in the stereo node).
  • [ ] Support a wide variety of color space conversions in Image Manip Node
  • [ ] Improve setKeepAspectRatio(False) behavior as per here

Luxonis-Brandon avatar May 19 '21 02:05 Luxonis-Brandon

A combination of distortion correction, crop and concatenation would be really nice for simple cases of image stitching.

jlecoeur avatar Aug 25 '21 18:08 jlecoeur