RFE: Add a non syncing key space and a ephemeral key space

[arv]

We should provide a way to store data in Replicache that is not synced.

It would also be nice to have ephemeral (in memory, not shared between tabs) key space for things like selection state etc. That way the application can use the same paradigms for everything.

One strawman is to have specific key spaces in the key value store that behaves differently.

[whiten]

Just want to add that this is added it may be useful to allow mutators to be tagged as local-only or ephemeral-only so they don't need to be kept around and sent to the server. If this was done, we should be sure to visibly error on any attempted writes outside the annotated bounds to flag bad annotations.

[arv]

We can detect what keys a mutation touches and based on that determine if it needs to be part of the push.

But that does raise a valid point that I did not think about before.

This needs a design doc. Maybe there is a better approach than key spaces?

[aboodman]

One potential use of this feature I have in mind is to maintain custom indexes. For example with https://fusejs.io/, one could implement FTS in userspace with this system 🤯.

[aboodman]

Some constraints in the design space wrt unsynced (I'll call it "local" from here on out) storage:

• people will frequently want to modify local storage transactionally with synced storage
• every single keywise api that exists in the system needs to exist for local storage, incl:
  • put, get, scan (plus all the features of scan), del

The best design is @whiten's original design where a subset of the keyspace is designated as local, e.g., /l/*. Unfortunately this is not possible backward compatibly as we did not reserve any characters of the keyspace or setup any way to do namespace :-|.

An alternate option would be to accept an array everyplace a key can be specified, e.g., get(["local", "foo"]) can be specified in addition to get("foo") or get(["synced", "foo"]).

[aboodman]

Yet another alternate option would be to have some separate interface on {Read|Write}Transaction like:

addPerson: async (tx: WriteTransaction, args: Person) => {
  await tx.put(args.id, args);
  // covering index on age
  await tx.local.put(`byAge/${args.age}/${args.id}`, args);
},

local would have the entire ReadTransaction/WriteTransaction interface except not clientID.

[arv]

I don't like the array idea.

I kind of like the .local idea but the type defs gets a bit weird:

interface WriteTransaction {
  readonly local: Omit<WriteTransaction, 'local' | 'clientID'>;
  ...
}

[whiten]

Regarding a designated keyspace, could a reserved prefix be declared (or opted out of) in ReplicacheOptions? In the 9.* series you'd probably want to make that setting required to use the feature, but either with the 10.0 release or an "options version" setting you could default to /local/ or whatever if not specified.

[aboodman]

These are good idea, thanks @whiten .

Maybe the thing to do is have a enableLocalKeys option in ReplicacheOptions and when enabled, the synced keyspace is prefixed with /s/ and the local features are available. When not, the keyspace behaves as today. We print a warning or whatever about enableLocalKeys becoming the default over several versions and eventually flip it?

[aboodman]

I wouldn't want to let users choose their local prefix because I don't get the point of that. Why would anyone care to configure that vs us just choosing that it's /l/?

[arv]

Sounds reasonable. Opting in to it for 10.x and log warnings if not opted in and key starts with the prefix and then in 11 or 12 make it default.