rerun-io
Manipulate the visibility/properties of a "Group" of Entities
Apologies if this is a duplicate.
Debugging motion planning workflows are quite unergonomic because there are no convenient ways to show all geometries of a certain "type" (e.g. visual, collision, inertial), or to manipulate the visibility/properties of those geometry types.
Description
Consider the following case where we log visual and collision geometries to every node along the following entity tree:
/a/very/deeply/nested/transform/tree/a/very/deeply/nested/transform/tree/maybe/a/very/deeply/nested/transform/tree/with/a/very/deeply/nested/transform/tree/some/branches
So, for example, the visual geometries might be in:
/a/visual/geom_id/a/very/visual/geom_id/a/very/deeply/visual/geom_id- ...
And the collision geometries:
/a/collision/geom_id/a/very/collision/geom_id/a/very/deeply/collision/geom_id- ...
The Rerun entity tree model is unfortunately not very well suited to isolating the "geometry tree", since the geometries will be distributed amongst the leaves of the entity tree.
The Problem
Now consider how it would be like to "toggle" visibility to see only all the visual geometries vs only all the collision geometries. To do this, we either need to:
- Generate a blueprint view for each of the geometry types programmatically where the entity filters exhaustively list out the path of each geometry entity (then the user switches which view they are using to see the appropriate geometry type)
- Exhaustively go through the entire entity tree to toggle the visibility of appropriate geometries one by one
Iirc, for (1) the global matching to be able to toggle something like /**/collision/* doesn't work yet, which necessitates exhaustively listing out each geometry entity leaf path.
All of the current options are very unergonomic, and borderline infeasible if done purely in the viewer UI.
Feature Request
Two options.
Alternative One
An alternative to this that I would prefer, would be to be able to annotate geometry entities with a "group" so that the entire group can be toggled at once. E.g. the visual geometries can be grouped under a "Visual" group and likewise for the collision.
Then there can be a separate UI for toggling properties/visibility of entire groups of entities instead.
Consider how RViz does it
Another Alternative
Alternatively, if we could somehow "extract" the geometries into their own tree to more easily manipulate, that could also work.
Great writeup, thank you so much!
Iirc, for (1) the global matching to be able to toggle something like /**/collision/* doesn't work yet, which necessitates exhaustively listing out each geometry entity leaf path.
We caught this idea here already, but without all the nice illustrative usecases and the group idea.
- https://github.com/rerun-io/rerun/issues/6673
Not mentioned there, but floated internally is also the extension of having even more powerful expressions that can be conditional on things like "has a mesh".
The alternative of introducing a grouping concept sounds super useful for me even beyond the problem described here: tagging entities across the tree can be useful for visualization as well! Making queries or override statements about everything sharing a tag/group is the logical next step then!
Hi there! One thing that already exists in our ui is an ability to search and filter the list of relevant entities via "keywords".
So in your example I could search for visual geom_id and see the list of entities including only visual geometry even if they are "differently" nested. Same would work for searching for collision geom_id that would list all entities with collision geometry.
Let's say you hit few results, then you may shift/cmd select them and .. ideally you would want to hide everything else, which is not yet there, but I have created an issue for it, which seems a faster way to get where you'd like (compared to the other ideas in the thread)!
tiny example of how I can search for parts of the entity path:
Hi there! One thing that already exists in our ui is an ability to search and filter the list of relevant entities via "keywords". So in your example I could search for
visual geom_idand see the list of entities including only visual geometry even if they are "differently" nested. Same would work for searching forcollision geom_idthat would list all entities with collision geometry. Let's say you hit few results, then you may shift/cmd select them and .. ideally you would want to hide everything else, which is not yet there, but I have created an issue for it, which seems a faster way to get where you'd like (compared to the other ideas in the thread)!tiny example of how I can search for parts of the entity path:
Well, that is pretty nice, but our smallest robot is already pretty unwieldy in terms of the transform tree (because of the entity tree model that Rerun has).
(And that's not even the full tree yet.)
yep.. it's still going...
Well, that is pretty nice, but our smallest robot is already pretty unwieldy in terms of the transform tree (because of the entity tree model that Rerun has).
WOW! So grateful for you to share this! We were considering to add a "flat" mode next to the "tree" mode.
With an idea that this view will give a better overview of "results" when the tree ends up being so long and cumbersome. something like this:
Curious to hear what's your reaction to this direction? Thanks! Ping @pweids
This appears related to an issue I filed recently about hiding collision meshes from URDF files, for a very similar reason: https://github.com/rerun-io/rerun/issues/10328
The hugely nested tree produced by the URDF also motivated some of the flattening UX that @gavrelina was discussing.
Well, that is pretty nice, but our smallest robot is already pretty unwieldy in terms of the transform tree (because of the entity tree model that Rerun has).
WOW! So grateful for you to share this! We were considering to add a "flat" mode next to the "tree" mode. With an idea that this view will give a better overview of "results" when the tree ends up being so long and cumbersome. something like this:
Curious to hear what's your reaction to this direction? Thanks! Ping [@pweids](https://github.com/pweids)
Well, it still wouldn't work as well, because the fully flattened form is still too long to even fit in the width of my screen.
For context, it is:
/viz/scene/tf/root/enclosure/base_link/robot/base_link/shoulder_pan_joint/shoulder_link/shoulder_lift_joint/upper_arm_link/elbow_joint/forearm_link/wrist_1_joint/wrist_1_link/wrist_2_joint/wrist_2_link/wrist_3_joint/wrist_3_link/tip_joint/tip_link/xxxx_flange_to_xxxxxxxxxxxxxx_adapter/base_link/tool_attach_link/ati/base_link/attach_joint/tool_link/ati_axia80_m20_to_xxxxxxx_pinch_gripper_adapter/base_link/tool_attach/xxxxbot_gripper/xxxxbot_vacuum_robotside/xxxxbot_vacuum_robotside_toolside/xxxxbot_vacuum_toolside/gripper_spacer/suction_spacer/suction_spacer_suction_cup/suction_cup/geometries/visual
Hey @gavrelina / @pweids , just checking if the additional information I provided up there ^ was helpful?
@methylDragon apologies for not following up on the above! It definitely is helpful (especially from a point of view of the real data). We did not move further with the grouping/tagging entities as of now, but we are also working on this issue https://github.com/rerun-io/rerun/issues/11174, which may be a way to simplify the hierarchy of the messages? Of course the fact that entity path may still end up way too long is an arbitrary issue. But hoping that it may significantly reduce the hierarchy! Let us know your thought, thanks!
@methylDragon apologies for not following up on the above! It definitely is helpful (especially from a point of view of the real data). We did not move further with the grouping/tagging entities as of now, but we are also working on this issue #11174, which may be a way to simplify the hierarchy of the messages? Of course the fact that entity path may still end up way too long is an arbitrary issue. But hoping that it may significantly reduce the hierarchy! Let us know your thought, thanks!
Ahh, that's unfortunate.
My current use case is making grouping entities even more important :(
Consider the following example:
- I am visualizing visual geometries, collision geometries, robot paths, and robot markers all in the same 3D scene, all attached to different transform frames
- I would like the ability to toggle each one of these layers individually without needing to keep track of every individual frame that they are attached to
When I just had geometries I could fudge it a little bit by adding extra tabs where I precompute all the paths, but now with multiple layers I won't be able to permute the views.
@methylDragon you're right that it doesn't directly solve the grouping problem even if it simplifies the hierarchy. We're also exploring multiple blueprints per recording with the ability to switch between them. While still not the same as tagging, you could potentially set up different blueprint views (e.g., "Visual Only", "Collision Only") to switch between. Would you be interested in chatting more in-depth (like a call?) about your motion planning workflows once we ship some of this (also transforms)?
@methylDragon you're right that it doesn't directly solve the grouping problem even if it simplifies the hierarchy. We're also exploring multiple blueprints per recording with the ability to switch between them. While still not the same as tagging, you could potentially set up different blueprint views (e.g., "Visual Only", "Collision Only") to switch between. Would you be interested in chatting more in-depth (like a call?) about your motion planning workflows once we ship some of this (also transforms)?
Yes, let's chat!
I had considered using blueprint views, and they worked when I only needed to toggle one variable (collision Vs visual). My issue comes when there's multiple variables that could be permuted (e.g. to express "hide visual, show collision, hide markers, show robot paths").