How to obtain eye gaze distance

[HaozheQi]

Hi, I was trying to project the eye gaze data to image space. However, as said in HoloLensCaptureExporter, the eye gaze extracted by HoloLensCapture is only represented by a 3D ray. However, in the original HoloLens2ForCV, they also use surface mapping to compute the eye gaze distance. I wonder if it is possible to also extract eye gaze distance in HoloLensCapture. If not, then do you have any idea about how to obtain this distance offline?

[sandrist]

I would try to compute the intersection of the gaze ray with the depth image meshes produced by the depth sensor. The "DepthImageCameraView" stream emits depth images along with the world pose of the camera and the intrinsics parameters.

Take a look at this IntersectLineWithDepthMesh to get an idea for how this can be computed in \psi (using a Line3D rather than a Ray3D, but it's easy to create one from the other).