Rerun should be the obvious replacement to imshow, plot, and draw_geometries

[nikolausWest]

The next major push will be towards making Rerun the obvious replacement for imshow (cv2, plt, px), plt.plot, and draw_geometries (open3d), for researchers and developers building algorithms and systems for computer vision an robotics.

In short this means you'll also be able to use Rerun as your visualization library for inline visualizations in scripts and notebooks. You'll have control over styling, layout, and other options directly from the SDK. Rerun should be usable both as your central visualization tool, and as the thing you use for quick hacks and experiments on the side.

This doesn't mean we'll necessarily replace every single feature supported by those function calls, but that when developers want to quickly visualize computer vision or robotics data in 2D or 3D, Rerun should be the obvious choice.

How?

• It will be at least as easy to do the simple stuff like showing an image, scatter plot, or point cloud.
• Powerful interactions for inspecting data are turned on by default.
• Complex visualizations are simple to make because things like label maps, 3D->2D projections, and time are core features to the data model.
• It will run equally well in distributed systems, standalone scripts, and notebooks.
• It will be significantly faster than the alternatives.

Why?

We believe we can make day to day work significantly easier for researchers and developers by upgrading these core visualization tools.

We've always focused on making it fast and easy to build powerful 2D and 3D visualizations. In the longer run Rerun will support you from the earliest prototyping all the way to production. However, everything starts with the day to day work of researchers and developers. That's where the requirements for ease of use, low friction, and minimal mental overhead are the largest. It's also where all projects start.

What needs to happen?

Rerun renders beautifully on the web and inline in Jupyter notebooks

• With full interactive functionality
• In both dark and light mode
• With minimal visual noise for single view visualizations

Visualizations are more configurable and controllable from the SDK

• All relevant blueprint (configuration) options should be settable both through the SDK and in the Viewer
• More available visualization primitives
  • 3D: cylinder, sphere, capsule, text, ...
  • 2D: circle, cross, box, line strip, ...
• Grids, axes, and legends
• Better more declarative layout system for multiple views
• Explicit control over the view's eye
  • 3D: lookat, fov, zoom
  • 2D: pan, zoom, crop
• Colormaps for images and tensors

Even more computer vision support in core data model

• 2D transforms (homography, crop, ...)
• Correspondences (keypoint matches, ...)

[asuper0]

Hi, these improvement would be great! I also have a question about current version, does there any solution to:

1. layout the viewer by code,
2. or lock the layout when restart my program.

[nikolausWest]

@asuper0

layout the viewer by code

Controlling layout with code isn't currently possible but is part of the work to finishing this big project.

or lock the layout when restart my program.

Layout and other options are currently persisted locally (between opening and closing the viewer) for a given application id. So make sure to set the application id when calling init method if you want that behaviour.

[asuper0]

or lock the layout when restart my program.

Layout and other options are currently persisted locally (between opening and closing the viewer) for a given application id. So make sure to set the application id when calling init method if you want that behaviour.

That works. I missed this becauce the recordings list in menu are displayed with UTC time, not local time. So I add a time string to the app id which may help me to identify different records. I deleted the time string in app id now. However, wolud be suffer with the UTC time🤣. Expect a setting about this.

[nikolausWest]

@asuper0

That works. I missed this becauce the recordings list in menu are displayed with UTC time, not local time. So I add a time string to the app id which may help me to identify different records. I deleted the time string in app id now. However, wolud be suffer with the UTC time🤣. Expect a setting about this.

Got it! Added an issue:

• #1714

[Atif-Anwer]

Is there any timeline as to when there will be any alpha/beta versions of this version or release for testing?

[Wumpf]

@Atif-Anwer we're releasing new versions every 2-4 weeks off our main branch. Not everything new landing in there is all towards the goals Niko layed out in this issue, but it is our main focus. We're expecting the next major release (in about 2-3 weeks) to make big strides in configurability & the 2D views.

Is there anything in particular you're missing from the latest release for your usecases? :)

If you haven't already, you can check our docs on how to get started with the latest stable release for Python and Rust.

[Atif-Anwer]

Yeah I really like the development progress on Rerun and really looking forward to the new bi-weekly updates :). As for the feature set, It would be really nice to have rerun as the defacto replacement for matplotlib as it would really help me with my work. Right now i am using the log_image and log_tensor as a replacement and it does work reasonably well.

[mvarble]

Is there any plan to have something like a 2D Plot view, where there are axes and interactivity akin to the Time Series view? I see that this merge means that v0.16.0 will have panning and zooming in the 2D Spatial view (which is great), but it still doesn't really apply to my use cases, where I would like to render a frame of radio/audio samples at each time, and the axes will scale dependent to the data.

I have attached an example from matplotlib to give visual reference.

Ideally, I would still log a LineStrips2D for each time, but I would be using a different view of those components. Alternatively, the 2D spatial view could have more blueprints to "lock" the axes to the data, render axes, etc.

[Wumpf]

Yes, we want to add non-time based plots eventually! Probably as customization options on the 2d view.