y-sweet should not 'sync' until data is persisted to s3
We've had at least one report from a user of data loss after using our application for several hours. I suspect there is some rare case where the data is no longer able to be persisted to s3, and this silently fails. For now, in our app, I think I can periodically query the s3 bucket for their project and check the timestamp to see if data is beyond some threshold, but there needs to be instead some robust way to ensure the y-sweet state isn't considered "synced" until written to s3.
I think the relevant code is at: https://github.com/jamsocket/y-sweet/blob/e13f6ced0d0bd22a2ff7f6fd6d5b4365f1ab88b1/crates/y-sweet/src/server.rs#L279
This is more or less my 'workaround' to at least alert the user and prevent wasted work:
function onUpdate(_: any, origin: number) {
if (!yDoc) return;
if (origin !== yDoc.clientID) {
lastServerUpdate = new Date();
if (lastS3Update && lastServerUpdate) {
// If the S3 update is more than 20 seconds older than the server update,
// we need to throw an error or block more user edits.
if (+lastS3Update < +lastServerUpdate - 20000) {
console.log("S3 is out of date, fetching from server");
}
}
}
}
😬