Lens Distortion Post Process : Wrong Labels

[ChumbaWambalala]

So I'm trying to replicate a cctv camera look with fisheye effect. I've set the FOV of the camera and added a lens distortion post processing stack. It turns out that the keypoints positions are not correctly compensated with the effect. Both position and occlusion. Project is in HDRP btw. I checked the output if it was only the labeler visualizer in unity having a problem, but its not. Coordinates were wrong on the final output with lens distortion.

Without Lens Distortion

With Lens Distortion

Would love to hear a better implementation or any workaround.

Thank you!

[mkamalza]

This is currently a shortcoming of our tools with regard to the Keypoint Labeler. While other labelers are compatible with lens Distortion, the screen space to world space function that we use to detect where each keypoint is in the output image does not take into account the active post processing effects. Currently, the only workaround would be to not use the lens distortion effect provided in Unity, and instead perform a manual distortion pass on the output dataset, which would post process both the RGB images and the keypoint coordinates.

[ChumbaWambalala]

This is currently a shortcoming of our tools with regard to the Keypoint Labeler. While other labelers are compatible with lens Distortion, the screen space to world space function that we use to detect where each keypoint is in the output image does not take into account the active post processing effects. Currently, the only workaround would be to not use the lens distortion effect provided in Unity, and instead perform a manual distortion pass on the output dataset, which would post process both the RGB images and the keypoint coordinates.

Thank you @mkamalza. Lens distortion works on other annotations but not on keypoints got it.