yjs
Emit event when missing structs to apply entire update
Is your feature request related to a problem? Please describe. We're syncing yjs documents over the network. Sometimes a message is dropped and the corresponding update does not reach a party. All following updates cannot be applied and the changes from one party are not synced. We could perform periodic resyncs - but if the interval is too big the parties would still be out of sync for a while. If it's too short the overhead of resyncs becomes too big.
Describe the solution you'd like If yjs emitted an event whenever an update cannot be applied because a previous update is missing we could use that to trigger the resync when it is needed.
Describe alternatives you've considered Being able to check if all updates have been fully applied on the ydoc would be another option.
Having both combined would allow for the following:
- listen for the event to see if an update cannot be applied
- wait for a few seconds in case messages just arrived out of order
- check if all updates have been fully applied
- resync if not
Additional context
We're syncing yjs docs via a legacy protocol that uses http requests. There's a lot we can do to make this more reliable and better suited for yjs. But for the time being a way to detect out of sync situations would be really helpful.
It looks like after an update that could not fully be applied ydoc.store.pendingStructs contains the structs that could not be applied. It's null otherwise.
Such an event would not reliably tell you that an update is missing. If an update does not depend on a missing update, then it would not be added to pendingStructs.
I'm not one for implementing "quick fixes" that just lead to more complex bugs in the future, if that makes sense. I also don't want to advertise going for quick fixes.
It's good that this issue happened, because now you know that your communication layer in unreliable..
You should make sure that all events eventually arrive. If your communication layer is unreliable, it could still be ensured by implementing a simple protocol layer on-top (e.g. a message counter could tell you that a message is missing). Alternatively, you could initialize a full sync with state vectors every x seconds. Or you could compare snapshots to check if the client has the latest content (if two Y.snapshot(ydoc) equal, then the docs are fully synced).
Sorry, this is probably not the answer that you were hoping for..
FYI, huge fan of Nextcloud! Happy that you are using Yjs over there :)
