couchbase-lite-core
couchbase-lite-core copied to clipboard
couchbase
Make sync work across VV upgrades
The transition from tree-based revids to version vectors is tricky for replication, and the prior approach (adding a fake 'legacy' version when creating the vector) didn't turn out to work.
Instead, now:
- when a doc is upgraded in memory to a VectorRecord it keeps its tree-based/legacy revids.
- When the doc is saved, it creates a version vector for the local Revision but also remembers the prior legacy revid.
- Remote Revisions keep their tree-based IDs until they're replaced by newer VV revisions (pulled or pushed).
As a result, revision histories now consist of a VV and/or legacy revids. The version vector, if present, always comes first. The legacy IDs at the end are what allow the recipient of a revision (an active or passive puller) to match up an incoming version vector with a local legacy revid.
(It's also possible for a rev history sent by the replicator to consist only of legacy IDs, even after the upgrade. This happens if a doc was updated locally before the app upgrade but not pushed until after. It can also happen on a pull for the same reason.)
There's an open question of how long we need to keep the legacy revid of the document. It only needs to be used once per remote, but in a P2P scenario you never know if you might have to sync with some new peer that hasn't upgraded that doc yet. In any case the revid isn't very big (~24 bytes) and it only occurs in pre-existing docs, so it may not be worth removing.
Code Coverage Results:
| Type | Percentage |
|---|---|
| branches | 65.9 |
| functions | 78.75 |
| instantiations | 33.18 |
| lines | 77.68 |
| regions | 73.25 |
The test Multiple Collections Incremental Revisions failed in mode PUSH and PULL:
2023-11-17T18:45:57.6745060Z /Users/runner/work/couchbase-lite-core/couchbase-lite-core/Replicator/tests/ReplicatorCollectionTest.cc:590: FAILED:
2023-11-17T18:45:57.6747260Z CHECK( _statusReceived.progress.documentCount == expectedDocumentCount )
2023-11-17T18:45:57.6748120Z with expansion:
2023-11-17T18:45:57.6748470Z 19 == 20
2023-11-17T18:45:57.6749250Z with message:
2023-11-17T18:45:57.6749710Z The replicator is forced to stop after timeout
This appears unrelated to what I've changed; it doesn't look like version vectors were involved.
yes, that test is flaky. It seems to happen when the replicator goes to Idle before all revisions get synced.
"when a doc is upgraded in memory to a VectorRecord" -- I noticed that method "upgradeVersioning" is deleted. When actually a doc is upgraded in memory?
Reviewing the code changes from looking at the diff here is hard for me to understand all of upgrading and handling both types of revisions. I'm still working on it. As of now, I have a few questions in general:
when a doc is upgraded in memory to a VectorRecord it keeps its tree-based/legacy revids.
Do I understand here that if the doc didn't get upgraded to vv, the vv record will be generated when read (but will not be saved)?
(It's also possible for a rev history sent by the replicator to consist only of legacy IDs, even after the upgrade. This happens if a doc was updated locally before the app upgrade but not pushed until after. It can also happen on a pull for the same reason.)
Do I understand correctly that when pulling a new revision that only contains the rev tree, the doc will be upgraded to the version vector (with the old rev tree kept) at that point including the new doc?
"when a doc is upgraded in memory to a VectorRecord" -- I noticed that method "upgradeVersioning" is deleted. When actually a doc is upgraded in memory?
Any time it's opened by the current version of LiteCore; the DocumentFactory is now VectorDocumentFactory.
Do I understand here that if the doc didn't get upgraded to vv, the vv record will be generated when read (but will not be saved)?
Yes.
Do I understand correctly that when pulling a new revision that only contains the rev tree, the doc will be upgraded to the version vector (with the old rev tree kept) at that point including the new doc?
The doc gets saved as a VectorRecord in the new format, but no version vector is generated since the new revision has a 'legacy' revid.
Please review the last commit, which is a change I had to make to get a unit test pass. I think the code in the test was wrong.