depthai
depthai copied to clipboard
Ability to return location in pixel space or xyz of generic depth blobs above certain size
Start with the why:
In many applications that involve obstacle avoidance, it doesn’t matter what the obstacle is, i.e. the obstacles encountered are not conveniently in a PASCAL-VOC-2007 20-class set, and in fact, the type of obstacles to be avoided could effectively be semi-infinite. So using a trained neural network for such obstacle detection may not be applicable in this case (as the training data may not be sufficient to detect a semi-infinite number of obstacles).
So enable use of DepthAI to solve such obstacle avoidance problems, one technique is to be able to mark an obstacle which is above a certain threshold above a plane.
This is then applicable to many common robotic navigation cases where for the most part the robot is/should-be operating on a plane (sidewalk, road, warehouse aisle, grass field, etc.) and it is desirable for the robot to remain in the planar areas and detect objects (and/or voids, i.e. holes) which deviate too much from this plane.
Move to the how:
This feature is likely best implemented leveraging the coming IMU support addition to DepthAI (https://github.com/luxonis/depthai-hardware/issues/8).
So in this way, the IMU can assist in plane estimation in conjunction with computer vision. Upon implementation, a literature review for plane detection and obstacle-above-a-plane detection is likely in order.
Move to the what:
- Plane detection
- Capability to mark something above a volume-threshold/height-threshold above plane as an obstacle.
- Capability for DepthAI API to set the volume and/or height-threshold.