storehaus icon indicating copy to clipboard operation
storehaus copied to clipboard

Make Paging stores easier

Open johnynek opened this issue 11 years ago • 2 comments

trait PagingReadableStore[K, V] extends ReadableStore[K,Spool[V]]
// see: https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/concurrent/Spool.scala

//combinator:
object PagingReadableStore {
  // C is some cursor type
  def from[K,V,C](readable: ReadableStore[(K,C), (V,C)])(nextPage: C => C): PagingReadableStore[K,V] // do the get, make an item for the Spool, on .tail, do the next call
}

This would be good for a store of Uid -> Followers on Twitter. Some have millions and we can't fit them into one response.

johnynek avatar Nov 16 '13 21:11 johnynek

Sounds great. I think this will fit in really well with Redis list and set stores. Can perhaps be done on MySQL as well using query LIMIT and some deduction for the cursor values.

rubanm avatar Nov 22 '13 06:11 rubanm

Yeah, the MySQL version would be great too. Indeed.

johnynek avatar Nov 22 '13 19:11 johnynek