"syncable" storage?

[inexorabletash]

We've been discussing what it would take to allow browsers to sync storage across devices.

Note that this is NOT the scenario where a web app wants to manually sync data to its own servers (or via some CORS-enabled cloud storage), but where the browser would silently replicate offline data across devices, much as Chrome replicates bookmarks and credentials. (Under user consent, of course.)

This would be challenging today as we'd probably need to sync an entire origin's storage as one monolithic entity to avoid breaking web apps that expect consistency. We also can't tell if what's stored in a database is ephemeral data (e.g. cache), data that can and perhaps should be replicated by the app already (e.g. pinned music, videos), or data that should "roam" (e.g. saved game state).

One initial thought is that you could make "syncable" an attribute of boxes. Since initial access to a box is async this also means we can defer access to a box while replication is happening. A user agent could also impose a limit on the size of a syncable box.

[annevk]

You could maybe hang this off persistent combined with having multiple boxes. But that would not enforce a limit and would not help you if you try to design a dialogless experience.

[wanderview]

Does "syncable" imply transferring usage stats? For example, if the box is used heavily on desktop, would it get bumped to the top of the mobile device's LRU?

In general, do we want to spec service oriented things like this in the browser? Would it not be better for the browser to provide some kind of UX to signal origins to sync/not sync?

[annevk]

I still think the best solution here is to just synchronize everything. It's a lot of data, but that's likely what most users want.

[inexorabletash]

The concerns about syncing everything are performance and reconciliation. On the performance front we probably don't want to sync storage which is effectively "cache". On the reconciliation front, if we sync everything then either we block access to storage while sync is happening or we define it for all storage types. We'd need to do that for cross-device sync anyway (even if it's just "most recent wins").

But we haven't thought deeply about this yet. Most of the developers we've talked with here want to be in control, and manage deciding what to sync / who the back end is / how reconcile changes themselves, and there's an ecosystem of providers like Firebase that give you an API for that. The scenario of "just make it work transparently" may turn out to be something we can handle entirely in the UA for sites that don't have a lot of data, using various signals.

[sicking]

Reconciliation really is the insurmountable problem. Syncing generally always requires require application-specific logic.

Lets take the most trivial imaginable. A consisting of 100 levels. The game stores a single numeric value, the level that user has reached.

Say that we simply sync the last updated copy of the full database.

1. The user play the game on their phone and reach level 10.
2. This value gets synced across all devices.
3. The user then play the game on their tablet while the tablet is offline, reaching level 20.
4. The next day the user opens the game on their phone again, which is still at level 10 since the tablet is still offline.
5. The user immediately realize that the progress from the tablet has not yet synced and so close the game. However the game always flushes to disk whenever closed and so the value 10 is written to the database.
6. The user opens the tablet and connects it to the internet so that it can sync, hoping to sync the progress across all devices.
7. Instead the value 10, which has a later modification date, is synced to the tablet.
8. User is sad because game is now back to level 10 everywhere.

Even in this extremely trivial example, where the database consists of only a single value, application specific logic is needed in order to know that the highest value is the right one, not the latest value.

There's really no way to do this automatically. Consider if the user decided to play one level in step 5 rather than closing immediately. There's no way to do this per-box or per-database in order to simplify the problem to the point that it can be handled by the UA.

If data has been modified in two locations we simply have to expose both the server-side data and the local data to the webpage and let it decide how to merge the two.

In practice I would also expect performance to make it impossible to sync automatically.

There are various solutions to this problem. With different APIs exposed to the client, and with different amounts of flexibility on what data can be stored and how.

I'd definitely think that we should reach out to people that have worked on database syncing. This is a really complex area and we should not invent our own thing, but rather leverage experience that already exist.