Should we have multiple readwrite interfaces that capture different consistency models?

[Mossaka]

Originally posted by @Mossaka in https://github.com/WebAssembly/wasi-keyvalue/pull/30#discussion_r1432038530

[dicej]

IMO: yes, we should. I'd suggest that the "primary" interface should have a relaxed -- but still useful -- consistency model, plus at least one more interface that provides strict serializability (i.e. linearizable and serializable).

I also think we need to think carefully about what the consistency model for the "primary" interface should be -- and document it clearly. Just saying it's "eventually consistent" has the virtue of giving exporters (e.g. hosts) maximum flexibility, but at the cost of being too weak for many applications. For example, if I call set twice and then get twice using the same connection, the results of the gets could be anything if there are no linearizability or "read-your-writes" guarantees. I could receive the latest value in the first get and then an old value in the second one, or some value unrelated to either what I wrote in either set. And with no way to block or be called back when values change, it's hard to imagine how one would write tests for an app that relies on such a weak consistency model.

That said, a very weak consistency model can still be useful for some apps -- we just need to decide how common those are vs. those which need stronger guarantees. Are we hitting 80% of use cases, or just 20%?

[thomastaylor312]

I very much agree with the statement of hit the 80% use case. Now that I think back on what I have used and see used, it might be best to say that the level of consistency any one of these should guarantee is "read-your-writes." I am curious if that is other people's experience as well.

As for providing another consistency model, I don't think I am in favor of this for the first go around. But I wanted to clarify what you meant here @dicej. Were you meaning that something like the "batch" or "atomic" interfaces would count as the secondary interface, or were you thinking of supporting another consistency model? Totally in favor of the former, but have some hesitations on the latter (which I can address depending on your answer)

[dicej]

As for providing another consistency model, I don't think I am in favor of this for the first go around. But I wanted to clarify what you meant here @dicej. Were you meaning that something like the "batch" or "atomic" interfaces would count as the secondary interface, or were you thinking of supporting another consistency model? Totally in favor of the former, but have some hesitations on the latter (which I can address depending on your answer)

I meant something orthogonal to the "batch" and "atomic" interfaces, e.g. something like:

interface strict-serializable {
  /// Get the latest value for `key`, blocking if necessary until all writes up to this point are visible.
  get: func(store: borrow<store>, key: key) -> result<option<value>, error>;

  /// Set the specified value for `key`, blocking if necessary to ensure strict serializability with respect to other writes.
  set: func(store: borrow<store>, key: key, value: value) -> result<_, error>;
}

To be clear: I don't think this is a "must have", but there are some applications for which these semantics are required, and we'll need to decide if (and when) we should support them (and whether that should be in wasi-keyvalue or elsewhere).

[thomastaylor312]

Ok, then I think I am in complete agreement with you. I would love to address that need for another interface (and where it should live) after we get this first draft out

[dicej]

Works for me.

FWIW, I wrote about state management in the context of handling incoming WebSocket frames here, including a concrete example that relies on a strict consistency model. Some of the discussion is Spin-specific, but could easily be generalized to a wasi-cloud app that uses both wasi-messaging and wasi-keyvalue to provide real time communications capabilities. I think that's an important use case to keep in mind since it comes up in multiplayer games, real time chat and video conferencing, and event-sourced architectures.

[lukewagner]

First, just as a clarification, the proposal as-is already provides two distinct levels of consistency in different interfaces: atomic provides different consistency than eventual.

That being said, I do appreciate the argument that it's useful to reduce cognitive load when a new users shows up and wants to know "what should I use" and doesn't know anything about "consistency models". In this context, since all the common key-value stores I'm aware of seem to default to eventual consistency, I think the argument could be made that this is a reasonable default expectation and so we don't need to explicit call out "eventual consistency" in the name and thus we can give the eventually-consistent interface the most obvious "use me" sort of name.

As for what the ideal "use me" sort of name is: independently of this whole discussion, I realized that it's really nice when WASI interface names are nouns, since then when a component imports, say, wasi:http/handler, you can say "the component imports a WASI HTTP handler". In contrast, "the component imports a WASI key value... eventual" doesn't sound right. What you naturally want to say is "the component imports a WASI key value store", so, given the above, I think perhaps we could name our main "use me" interface wasi:keyvalue/store (instead of wasi:keyvalue/eventual).

Since the same argument applies against wasi:keyvalue/atomic (which is also an adjective), perhaps it should be named wasi:keyvalue/atomic-ops or wasi:keyvalue/atomics.

[thomastaylor312]

Thanks @lukewagner! I'll be working on opening a PR soon with some proposed changes and I'll fold this in

[thomastaylor312]

@Mossaka I think with the merge of #41 we can close this. Do you agree?

[Mossaka]

I prefer to leave this issue open since #41 only changes eventual -> read-your-write but the issue around interfaces that express multiple consistency levels is still a valid one and I hope more people could join the disucssion and provide feedback.

[lukewagner]

FWIW, we already have multiple consistency levels present (with store and atomics), it's just that we have a very limited set of operations on the latter. It is a valid question of whether we want to add more (levels or operations to existing levels), though.

[thomastaylor312]

I figured we definitely would have those discussions, but wonder if we should close this so people can open more scoped proposals later. I think the answer to this issue itself (Should we have multiple readwrite interfaces that capture different consistency models?) is "Yes!" And then we can open new issues when people are ready to add them.

Either way I don't mind, was just doing a bit of housekeeping