depthai-experiments
depthai-experiments copied to clipboard
luxonis
Gaze Estimation Bounding Boxes Left/Right eye drawings
The bounding box (possibly just the UI drawing) of the left and right eyes are not updated according to the right scale. Maybe it is also great to verify we are also passing the data correctly.
note. this issue is mentioned in https://github.com/luxonis/depthai-experiments/pull/51
Thank you for documenting this! I was wondering why the boxes are so big...
the bounding boxes for eyes are generated on the UI, so they can be changed. However, they are now reflecting what is being sent to the neural network. It accepts eye images scaled to 60x60, and from landmarks neural network I receive a single eye point, so using padded_point(..., padding=30, ...) is producing a correct bounding box that I can send further.
This is also why this bounding box is sometimes too big / too small - as it doesn't take the face dimensions into account.
One fix for this, that would be fairly easy to do, is to somehow correlate the face dimensions with eye size, so that for given face width/height we can estimate the eyes width/height. This way, we could draw a better bounding box and the scaled image for nn would contain less non-eye parts, so it may also improve efficiency
Sharing also here for readability
Drawing Bbox should have as main objective to be used as debugging. The gaze model requires a 60x60 px input for the eyes. Having too small or a large bbox, help understand why the model works well or badly. (If we are too close or too far).
To improve the cropped area (and thus have a better bbox for the eyes), we must find a way to adapt the resolution of the cropped face, using multiple resolutions/zoom for example ...
Changing the drawing of the bbox, to adapt well, change the effect and not the cause ... So, I agree with @VanDavv about it.
So am I right that currently we are feeding the wrong 'size' into the NN? I mean we don't take the face size into account?
Yes, rescaling for the near range (depending on input resolution) and using the zoom feature (not yet implemented / finalized) for far distances should improve the result.
Ideally, we would have access to the training dataset and try to crop it the same (for eyes) ... ;-)