Support for partial fetches (for paginated APIs)

[johnheroy]

Looking at the Fetcher interface, I'm wondering if there's been any consideration around supporting paginated APIs and provided some kind of param to fetcher based on what data is already saved in the disk cache (e.g., last updated timestamp)?

The kind of use case I'm thinking about if we are fetching some long list of events ordered by one of the response object's field (e.g., some timestamp) and there are lots of entries already on disk and we only want to ask the remote data source for the ones newer than the one we already have saved on the Android device.

[digitalbuddha]

Hi, I don't have anything that is refined enough to add to Store yet but I've been working on a pagination setup for work myself. Rather than having Store support pagination, I make a wrapper around a store to handle the pagination state:

class FeedPager constructor(val singlePageStore: ThreadStore) {
  val page = AtomicInteger(-1)

  fun nextPage(): Single<List<Thread>> {
    Timber.e("getting page ${page.get() + 1}")
    return singlePageStore.get(page.incrementAndGet())
  }

  fun allCurrentPages() = Observable
      .range(0, page.get() + 1)
      .concatMapEager({
        singlePageStore.get(it)
            .toObservable()
      }, 10, 10)
      .toList()
      .map { it.flatten() }
  //refresh calls store.clear then store.fetch so that we clear the memory cache
  fun refresh(): Single<List<Thread>> {
    return singlePageStore.refresh()
        .doOnSubscribe { page.set(0) }
  }
}

Since the persister will cache pages I already loaded, rotation allows me to start on page1 again and load from disk rather than network the 2nd time. Hope it helps :-)