Add .rrd files to `git lfs` to test backwards compatibility
We want to have automated tests to make sure we have backwards compatible chunk loaders.
Ideally we should add .rrd files to git lfs for all previous versions of all components and archetypes.
What to test?
- They load without error/warning
- Passes
rerun rrd verify- https://github.com/rerun-io/rerun/pull/9128
- Produce the same
rerun printoutput?- Maybe just the column headers (names, meta-data)?
- That's a useful sanity check when we update the .rrd:s in the test set, that the new ones covers approximately the same things
- Produce the same visuals? Difficult to get right
What .rrd files?
- [ ] Snippets?
- [ ] Examples?
- [ ] One .rrd file containing all known components/archetypes/datatypes?
When and how do we add more .rrd files to the test set?
What if something does break?
We do not yet promise full backwards compatibility, but any CI failure regarding this should be a red flag, and we should only accept it (and upload a new .rrd file) if we are really ok with this breaking change.
What remains to do?
- [ ] Decide on exactly what .rrd files to test
- [ ] Automatically verify that all snippets and examples etc have .rrd:s that are checked in
- [ ] Add .rrds under folders with version string
- [ ] Compare all versions of old .rrd files (e.g. from 0.22, 0.23, 0.24…)
- [ ] Ensure we add new versions of snippet and examples iff needed
- [ ] Improve the script for updating said .rrd files to:
- [ ] Not be a bash script (-> Python or Rust)
- [ ] Add missing files (e.g. when adding new snippets) without changing the existing files
- [ ] Update all existing files (when intentionally breaking backwards compatibility)
- https://github.com/rerun-io/rerun/pull/9370
Potential improvements
- [ ] Migrate the checked in snippet rrd files, then compare them to the output of the latest snippet
compare_snippet_output.py should compare its output with the .rrd files checked in to git lfs:
- If missing, it is an error (a new snippet needs to check in an .rrd for its output)
- If it differs, it is an error
Complications
If we change a snippet (so that it outputs something different), we need to either:
- Update the file on
git lfs - Add a new file to
git lfsalongside the old one - Add an opt-out flag for comparing to the file on
git lfstosnippets.toml
Since this should be a rather rare occurrence (our snippets tend to stay pretty stable), any one solution will do.