gardencmp
Feature Request: Rewrite History when Backfilling data
In the transition path to Jazz, we were thinking about backfilling data from a different database. Most other databases and frameworks keep track of updatedAt and createdAt. It would be fantastic if we could support mutating these values. When an activity log is already present, we could go a step further and think about backfilling historical data with the appropriate updatedAt timestamps as well.
I understand that this needs some extra care, since we don't want regular users rewriting history. Perhaps this could be paired with a super-user #1235 who can make these edits? Maybe there is another way?
Part of the design of Jazz is that the history can't be modified, you can only append new entries.
What we could do is to store additional metadata to enrich the current history, but mostly depends on your needs.
Do you need that to track the progress of backfilling the DB, or for having the history in the UI matching the data in your DB?
We want to preserve most of the data from our previous DB, if we transition.
I understand that it's core to Jazz that we can't change the data as a user, but maybe we can find a way during the backfilling process? Perhaps using a local sync node and a server worker, we could decrypt the sqlite database and update the history manually?
The timestamps are used for permission checks, so I wouldn't play with those because it would be very easy to mess things up.
What we could do is store in the data you are importing from the DB some metadata that says "this transaction happened in X" and when this info is not available we fall-back to Jazz timestamps.
So UI-wise you see the updatedAt/createdAt is consistent with your DB, but under the hood the Jazz timestamps are pristine.
Interesting suggestion. I like how the coValue UI would stay untouched with that!
This is a prime use case for bitemporal database schema. Usually it starts with "event date"/"record date" (logical/physical), where record is managed by the database system, and eg if events can be corrected/new info arrived then another record is inserted with a different record date but the same event date (or even a different event date if it is discovered that the date was initially recorded incorrectly)
For our use cases updating the createdAt in unique ended up being enough. There wasn't a need to also migrate the updatedAt. Closing this issue for now!