storehaus
storehaus copied to clipboard
twitter
Write-Only Store Abstraction
It may be convenient to have a built-in or explicit abstraction for stores where the read and write paths are distinct. Currently you can have a readable store but the next level of abstractions are Stores which readable and writable. It may be convenient to have a wrapping store that delegates all reads to a read-only store and all writes to a write-only store. I'm not sure what you would call this but it could just be a wrapper for both sides which delegate on puts/gets
// a better name maybe???
class PartionedStore(readStore: Store[...], writeStore: Store[...]) {
override def get(k: K) = readStore.get(k)
override def put(kv: (k: K, v: Option[V]) = writeStore.put(kv)
// yada yada
}
Related, it may be convenient to define an explicit write/append only store for things like Stats and logs.
If this is not worth while, close this issue out.
For network service backed stores that wish to do this, they can always define interfaces for supplying two underlying clients for store operations: one configured for reading and one configured for writing.
We used to have a trait called Mergeable that handled the write-only case, but we punted since it was unclear what sort of guarantees such an interface should provide -- I totally get the case of logging and stats, though. We have some ad-hoc stats wrappers inside Twitter, but we haven't really come up with a good interface for this yet.
Maybe we want something like
trait Sink[T] extends (T => Future[Unit])
That's a start. I'll think about it for a bit. Let me know if you have other ideas
I'm +1 on this thread. Especially on having a write-only abstraction - I miss Mergeable (but I think the semantics don't need to be that well-defined, ie, if it wants to merge that's fine but since it's write-only you don't need to worry about that high up the pipeline).
To be consistent with the naming ReadableStore, would make sense to just define a WriteableStore that defines put and multiPut methods?
We had played around originally with a Mergeable abstraction, but the meaning of such a trait was a bit fuzzy. If you can't read items back out, what's the point of having both K and V, for example?
What do you guys think of keeping the current hierarchy and adding in:
trait Sink[+T] extends (T => Future[Unit])
class SinkStore[K, V](readableStore: ReadableStore[K, V], sink: Sink[(K, Option[V])]) extends Store[K, V] {
def get(k: K) = readableStore.get(k)
def multiGet[K1 <: K](ks: Set[K1]) = readableStore.multiGet(ks)
def put(pair: (K, Option[V])) = sink(pair)
def multiPut[K1 <: K](pairs: Map[K1, Option[V]]) =
pairs.map { case pair@(k, _) => k -> sink(pair) }
}
// Another possible combinator:
class StoreWithSideEffect(store: Store[K, V], sink: Sink[(K, Option[V])]) extends Store[K, V] {
def put(pair: (K, Option[V])) = sink(pair).join(store.put(pair)).unit
def multiPut[K1 <: K](pairs: Map[K1, Option[V]]) = <etc>
}
I kind of get the terminology of Sink but the word Writable seems to be more inline as the dual vocabulary for Readable. In other words, I immediately understand that Writable is something I can write to the same way I immediately know I can read from something that's Readable.
I think there are cases for stores that you just write to, fire and forget services like logging and stats and other append only stores, that could make meaning of put and multiPut.
I think you can keep the Store interface as is but make it mix in a WritableStore trait which defines put and multiPut, and use that as its a base for extension for other compositions of both. Current clients wont need to change and you can start defining append only stores based off the new trait.
Agree with @softprops, but just to make a related point: I think the one big use case of this is for clients who only need a Writable to be able to re-use existing implementations of Store (without constraining themselves to only using things that also implement get, so that we can have a StdoutWritable etc too).
I think semantically they don't need a K, just a V representing the type they can write.
Semantically they don't, but I think that given the rest of Storehaus has a K,V, we should keep that here for consistency. I was imagining StdoutWritable looking sorta like this:
object StdoutWritable extends Writeable[String,String] {
def put(pair: (K,Option[V])) {
println(pair._1 + "\t" + pair._2.getOrElse(""))
}
}