Graph archetype

[vmayoral]

Most of the computations in robotics are expressed in the form of computational graphs. Whether you use ROS, ROS 2 or alternative frameworks, this seems to be the general consensus.

(an arbitrary ROS graph)

Besides their use for dev-purposes, these graphs are a fundamental tool when visualizing a robot's architecture. Most ROS-related graph representations are lacking. There's an interesting opportunity to get this right within rerun. That would make it much more useful for roboticists.

[nikolausWest]

Thanks @vmayoral!

There's an interesting opportunity to get this right within rerun.

could you expand on this? Sounds very interesting!

[vmayoral]

Hey @nikolausWest,

I'll try to expand on both the motivation and on the technical approach, sure:

• Motivation-wise: (As I believe you folks know already) Robots are networks of networks, with sensors capturing data, passing to compute technologies, and then on to actuators and back again in a deterministic manner. These networks can be understood as the nervous system of the robot, passing across compute Nodes, that represent neurons. Like the human nervous systems, real-time information across all these computational Nodes is fundamental for the robot to behave coherently. "Robot brains" are built with this same philosophy. Behaviors take the form of computational graphs, with data flowing between Nodes, across physical networks (communication buses) and while mapping to underlying sensors, compute technologies and actuators. Frameworks like ROS enables you to build these computational graphs and create robot behaviors by providing libraries, a communication infrastructure, drivers and tools to put it all together. Alternatives to ROS also bet into this graph-like concepts. In a way, robot behaviors derive from the dataflow across these graphs, thereby getting a good representation of such (computational) graphs is fundamental for robot visualization.
• Technically (getting this right[^1]): Let me start by telling you what works with limitations and what doesn't, and then throw a few suggestions.
  • Most of us still rely on rqt_graph for (robot) graph visualization. This tool hasn't evolved alongside modern software concepts in robotics (aspects such as ROS Components as opposed to Nodes (or Nodelets) aren't depicted). There're plenty of limitations on this regard. You still can't easily properly represent other-than-pub/sub relationships into these graphs, for example client-server interactions such as ROS services.
  • Web-based alternatives to rqt_graph that I know are very lacking. For example Foxglove's Topic Graph is useless, probably resulting from a lack of understanding of the criticality of these graphs for demonstrating/showcasing/building/repairing/improving/debuging robots. There're other web-based robot graph visualizations that have been prototyped, but they simply lack the simplicity, (programming) usability, easy of customization and responsiveness that you folks at rerun seem to be taking at your core.
  • Suggestions:
    • Graphs should be simple to visualize, responsive and easy to custom programatically.
    • It should be possible to represent basic pub/sub interactions between nodes in the graph as well as other communication paradigms, capturing abstractions like ROS services or actions.
    • Ease of graph customization is key for succeeding in my opinion. Modern/custom aspects such as depicting subgroups within the graph to capture Components vs Nodes, or inter-network vs intra-network communications within a robotic system come to mind as relevantly unmet.
    • There's a huge opportunity in rerun to make use of theSelectionPanel to further augment information by selecting a node in the graph. This information could/should bet set programatically within each graph node, and could be extracted/relied from data-frameworks like ROS. Things like frequency of interactions, bandwidth, latencies (max/mean/min), etc would be tremendously useful if used together with your already existing ploting capabilities (e.g. TimeSeriesView).

[^1]: I can only throw a few initial ideas on the getting right topic, as "getting it right" would likely require iterations and feedback for its achievement

[nikolausWest]

@vmayoral: we're considering giving this a go soon. Would you be willing to feedback on some early designs / requirements to help us make sure we actually get it right?

[grtlr]

I've written down a proposal for extending the data model to handle graphs here: https://github.com/rerun-io/rerun/issues/7431. This could be a first step to implement the use cases that @vmayoral describes here.

[grtlr]

Quick Update: With #7500 the graph primitives have landed in Rerun. The next steps are working laying out these graphs, so that they can also be used to visualize ROS node/topic graphs.

[simgt]

I'm using the graph view for a compute graph in gst-rerun. Some notes for potential future work on rerun's graph view:

• the force-based layout is obviously not suited to this kind of graphs, instead I set the positions of the nodes with a Sugiyama-style algorithm, it works well enough while being very simple
• it's fairly easy to make the layout algorithm stable across time for compute graphs, outputs are often fixed for a given type of node
• I'm missing the possibility of having "subgraphs" inside some nodes (which I have here in a separate app that could become a custom view in rerun if I find the time)
• having nodes being entities would be nice, as it'd allow linking them to other data points (in my case the ending nodes of the graphs are the ones sending VideoStream or Image logs to rerun)

I recently came across this article from spidermonkey that you may find useful for layouting compute graphs.

[grtlr]

That is very interesting feedback, thank you! The spidermonkey article is definitely on my reading list now 🤓