How to "implement" vertex selection using SceneWidget?

[anticdimi]

trafficstars

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

Hi, I would like to have the same functionality as VisualizerWithVertexSelection but using GUI classes. Is it possible to somehow alter the implementation shown in the example: examples/python/visualization/mouse_and_point_coord.py such that it returns the selected vertex on the mesh? I have tried using the naive approach by finding the closest vertex to the point that's obtained as coordinates world, but somehow I always get the wrong point. Moreover, even though I click on the mesh, I never get the point that's contained in the vertices set of the mesh.

[sadams207]

You may be experiencing something similar to what I found: https://github.com/isl-org/Open3D/issues/5534 Are you useing defaultLitSSR shader? If so, the mesh will not show up in the depth map and be selectable. Using defaultLit shader, it worked fine for me doing exactly what you said.

[anticdimi]

Hi @sadams207, I went through your code example and it looks like we are doing exactly the same thing. Have you checked if any vertex coordinates are equal to the clicked point coords? From what I have noticed, they are the same until the third decimal point, after that, in my case, coordinates differ. The naive workaround is to select the vertex closest to the chosen point wrt euclidean distance but it seems quite strange that there even exists such a discrepancy in coordinates...

[sadams207]

@anticdimi, the discrepancy may be because you are not really clicking on a vertex. Instead, you are clicking on a depth_map image. Points, therefore have a width and a discretization. The world coordinate computed from the depth_map is a little bit different from the underlying vertex. I did the same thing you suggested with the euclidean distance, but I had to expand it a bit because I mixed lines, mesh, and points in my scene.

[anticdimi]

Hi @sadams207, yeah your point makes sense completely. Now I am actually settling for a different solution, by using selection sets logic implemented in cpp/open3d/visualization/visualizer/O3DVisualizer.cpp as it is working pretty well and also having the polygon selection option, but modifying the source code for things I want to change in the layout of the window.