rerun-io
New API to request userspace-callbacks from interaction events in visualizer
A recurring request from users is a mechanism for user-interactions such as clicking to generate a callback in user-space code.
Some examples:
- Click on a point and receive the timepoint + entity-path + instance-id
- Show an arbitrary list in a UI element and receive an event from the selection
Alternatives:
- Until this feature is implemented, to accomplish something like this a user would need to create their own custom space-view: https://github.com/rerun-io/rerun/pull/2328 and then send data over a custom out-of-band backchannel, completely independent of Rerun.
TODO:
- More details on roadmap for this feature
Hey! I'd love this feature too. I went through the mentioned PR but it looks like it's meant for Rust and not Python. If there is a pythonic way to create custom space views, I'd appreciate any help. Thanks!
@jaidevshriram yes, the linked PR is sadly Rust only and this is a very heavy-handed way of extending the viewer. This ticket is indeed all about making smaller adjustments from all languages in a more data-driven way :) Unfortunately, this is a bit further down the line; we're first going to look into more basic control of the viewer layout from code (if you see us talk about blueprint in tickets and on discord that's that)
Hello,
If there is a way to expose a javascript library that will enable usage of rerun specific things with custom UI components, that would significantly help in using this tool for custom tools on top of the renderer/viewer etc.
Just echoing the need for this feature on my end as well. I'm making a 3D mapping system for robotics that allows you to search for objects in the scene via natural language, and this would help a lot.
See: https://github.com/concept-graphs/concept-graphs
My specific need is: Having something like a text input box with a labelled "search" button, that when filled out and pressed would trigger a callback to our code that performs the search and opens a new view with the desired object, or recolors all the objects in the scene by how relevant they are to the search query.
This would really help people migrate more easily from other options to rerun. Thanks!! Rerun is awesome.
+1 on user inputs, like buttons/text inputs/sliders (perhaps API could be similar to streamlit one?). OpenCV provides sliders, so I use opencv just for that, and then rerun for visualziation.
+1 this with python interfaces for mouse over and selection in 2D views (expanding graph nodes) as well as keyboard callbacks.
+1 for a callback API with extremely basic buttons, sliders, and text boxes. Even something as simple as lil-gui would be a gamechanger.
User input and multiple servers feeding one client are the two biggest barriers for me right now 😅
multiple servers feeding one client
We have that already! :) Two processes (no matter where) can feed the same Viewer by specifying the same appliation & recording id
General update: 0.23.0 will introduce some basic callbacks for the npm package, jupyter notebooks and self-hosted Rust builds of the Viewer.