rerun icon indicating copy to clipboard operation
rerun copied to clipboard

Move `store_source` and `store_version` from `StoreInfo` into `archetypes::RecordingProperties`

Open grtlr opened this issue 9 months ago • 1 comments

Observation: store_source and store_version are used mostly for analytics, so they are not critical from a backwards-compatibility perspective, but it would still be great to have this information when we transition to a new container format.

As discussed with @emilk the goal is to move over these two fields to RecordingProperties one-by-one. There are several considerations:

  • Adding new properties to RecordingProperties means having to adapt the rerun rrd compare tool to ignore additional columns.
  • From an analytics perspective, some details in store_source, like FileDialog and DragAndDrop are better captured when these events happen in the viewer.
  • We need to decide what level of analytics we want to carry over. New analytics events should probably be emitted when the columns with the relevant descriptors appear in the ChunkStoreDiff in re_chunk_store. Note that this will likely have an influence on the analytic metrics.

Since we don't have tagged components yet, we should probably end up with something like the following in recording_properties.fbs:

table RecordingProperties (/* ... */) {
  // ... existing fields

  /// An enum where the data came from (`CSdk`, `PythonSdk`, `RustSdk`, `File`, `Viewer`, `Other`)
  source: rerun.components.RecordingSource ("attr.rerun.component_optional", nullable, order: 3000)

  /// The Rerun SDK version.
  rerun_sdk_version: rerun.components.RerunSdkVersion ("attr.rerun.component_optional", nullable, order: 4000);

  /// The Python version (e.g. `3.11`, ...)
  python_version: rerun.components.PythonVersion ("attr.rerun.component_optional", nullable, order: 5000);
}

Both RerunSdkVersion and PythonVersion can just be wrappers around datatypes.Utf8. Note that we need these extra components because we don't have tagged components (#6889) yet.

There is a remaining field, cloned_from in StoreInfo that we never store in our files. Probably best to move it to a different place like EntityDb.

grtlr avatar Mar 05 '25 09:03 grtlr

Adding new properties to RecordingProperties means having to adapt the rerun rrd compare tool to ignore additional columns.

As mentioned on slack, I think we should split RecordingProperties (user-sourced) from something more specific like SDKSourceProperties

Regardless, is it possible for rrd compare to just ignore ALL __properties from a specific Archetype so that this doesn't need to be handled on a property-by-property basis?

jleibs avatar Mar 18 '25 12:03 jleibs