rerun-io
Python API to administer (remove) applications, recordings and entities
Is your feature request related to a problem? Please describe. Currently, it is not possible to administer applications, recordings, or entities via the Python API. When working with multiple recordings or wanting to update recordings, I would like the ability to remove them programmatically. At present, the only way to remove them is by manually clicking in the viewer. Additionally, when working with numerous recordings, the viewer may end up with partially deleted recordings, as it automatically drops the oldest datasets.
Describe the solution you'd like A few api commands to administer recordings.
Commands to get the current state:
rr.get_applications() -> returns all application ids
rr.get_recordings(<application_id>) -> returns all recording ids for the given application id
rr.get_entities(<application_id>,<recording_id>) -> returns all entity ids for given application and recording id
Commands to delete specific things:
rr.delete_application(<application_id>)
rr.delete_recording(<application_id>,<recording_id>)
rr.delete_entity_path(<application_id>,<recording_id>,<entity_path>)
Commands to delete more / all things:
rr.delete_all_applications()
rr.delete_applications([list of ids])
...
Describe alternatives you've considered There seems to be a pull request here: and an issue here but I can't find any documentation on drop_entity_path in the python api.
All these things exist in the viewer, so I hope this could be easy to implement.
@Wumpf : Just wanted to know if there is any plans in implementing anything like this?
It's not called out on any short term roadmap, but that said I think as we move to a client/server everything model we're getting a lot closer to a nice way of expressing this: as we're starting to build out our commercial dataplatform we also already (more or less stealthy) published an open source server that speaks the same protocol. The idea is that in the hopefully near future the Rerun viewer always connects to a server and then we can have operations on that server like removing recordings etc. Obviously, connecting to a local Rerun Viewer today isn't all that different from connecting to some server/store of recordings, but this has a lot to do with how we want to model additional data/viewer communication in the future, which is why I bring this up.
cc: @abey79 having a "remove recording" endpoint on OSS server should be pretty easy if we don't have it already, right? 🤔
cc: @abey79 having a "remove recording" endpoint on OSS server should be pretty easy if we don't have it already, right? 🤔
This points at one of the hurdles to use the server as viewer back-end: for the server stuff, we moved away from the notions of "application id" and "recordings". The closest matches are "dataset" and "partitions" (soon to be renamed "section"—yeah, much of this is still in flux). We can already remove entire datasets, but we currently don't have a "remove partition" operation. Currently, such an endpoint would be trivially implemented on OSS server. On Cloud, this might be slightly more involved due to all the indexing going on.
Is there any documentation on the server I could read? Seems like something i would like to use!
Is there any documentation on the server I could read? Seems like something i would like to use!
The documentation on this stuff if sadly very lacunary. The open source server is built-in the rerun cli now, so you can do rerun server --help. As for the Python SDK, the entry point is the CatalogClient class, which is documented here: https://ref.rerun.io/docs/python/0.27.0/common/catalog/#rerun.catalog.CatalogClient
Thanks for the link. As far as I can understand, the OSS server is there to provide files for the viewer? Or is there some more information on what it is for?
Thanks for the link. As far as I can understand, the OSS server is there to provide files for the viewer? Or is there some more information on what it is for?
This is not yet fully reflected by our landing page (update is in progress). The data model of the Rerun server (which we call "redap" for rerun data protocol), as well as the querying capability we're building in the SDK, are geared toward implementation robotics/physical AI data pipelines. The OSS server is a "basic" implementation of redap, with limitations in scaling and persistence (e.g. all data is held in RAM), but is otherwise fully functional and can serve with, e.g., local-scale pipelines, prototyping and CI/test automation. Our upcoming managed service will provide a scalable service over the same protocol.