webknossos icon indicating copy to clipboard operation
webknossos copied to clipboard

Versions of volume data should be created at "transaction" points

Open philippotto opened this issue 2 years ago • 3 comments

Currently, editing volume data will create new versions where each version consists of at most 32 changed buckets (see COMPRESSING_BATCH_SIZE). However, this means that reverting to an older version might very well restore a state where chunk borders are visible since the version was created right in the middle of saving. Ideally, the saving mechanism should use a transaction logic (similar to how it's done for skeletons) so that each version is a consistent tracing state.

An additional benefit of ditching the 32 limit is that we reduce the amount of versions being created (one large brush stroke can already cover much more than 32 buckets).

philippotto avatar Sep 26 '22 15:09 philippotto

Also see https://www.notion.so/scalableminds/Design-Doc-Versioning-Transactions-90f514760a87491699fffa3fe2ba602c#f39d5acdc22e4035aa96d6006b4528d4.

philippotto avatar Oct 25 '22 14:10 philippotto

@fm3 Are the transaction{Ids,groupIndex,groupCount} persisted or is it only ephemeral to ensure that no "half transactions" are stored? @daniel-wer and I noticed that the version history view doesn't make use of the transaction concept currently. If the properties are stored, we could use them to show semantic versions.

This would only help for skeleton tracings, though, since the front-end doesn't really make use of transaction groups for volume tracings.

philippotto avatar Oct 25 '22 15:10 philippotto

The transaction grouping is not currently stored in the db, it is only used to hold back its update groups until the transaction is complete, then they are all inserted into the db. We can of course add more optional fields to the stored update actions, encoding the transaction ids. But as you say, for volume tracings we would also have to change the frontend

fm3 avatar Oct 26 '22 08:10 fm3

Just for the record: We decided to establish a 1:1 relation between transactions and versions (see #7290). That way, we don't need to store additional properties, such as the transactionGroupIndex.

philippotto avatar Aug 29 '23 12:08 philippotto