Keyset pagination support

[karlbecker]

Hello!

Our API uses a keyset pagination method. On the response of each paginated endpoint, we return a key that indicates whether there is more data or not. Our consumer apps send the key back to the API, which tells the API to give me the next page of data starting with the data stored in the key.

This technique eliminates some of the trickiness such as data being updated in the middle of getting paging results.

Example to demonstrate the issue with a page number and page size approach:

• at 11:07pm, the app requests the first page with items 1 - 10
• at 11:09pm, something causes the server to insert a couple items at the top of the list, bumping down items 1 - 10 to now be items 4 - 13
• at 11:15pm, the app requests the next page with a fixed page size, and the API responds with items 11 - 20... but item 11, 12, and 13 were already sent in the first page

Couple questions:

1. Is there any way this either can already be supported, or any plans to support this, with this library?
2. Do you (the leadership team of this add-on) think this be able to be accomplished fairly straightforwardly with using Impagination directly?
3. Would you be interested in making this a standard feature of this add-on?

Thanks!

[Alonski]

Hey @karlbecker So one of my teammates ran into a similar issue using Google APIs that are paginated similarly to what you are describing. After a lot of fighting with Impagination we decided that it wasn't the right tool for the job.

Impagination is designed to request multiple pages simultaneously, not one by one while waiting for a response from the past one.

I assume with a little bit of hacking this could be possible. If you figured it out I would definitely welcome a PR. I don't really have any recommendations on how to do this though.

If you start something and have questions along the way feel free to ask and I will try to help to the best of my abilities :)

Good luck!

[karlbecker]

Thanks for the heads up, @Alonski , you probably just saved us a couple day code spike's worth of effort to try this library 😊

I'm surprised it does multiple pages simultaneously... I consider pagination to be central to reducing load on the server. Hitting a server with many page requests very close to each other might result in a good UX for the end user (as long as getting new data doesn't shift the scroll position around in an app!), but it could be rough on the server I suppose.

What solution did you end up going with?

[Robdel12]

Might be worth pinging @flexyford here too :)

[Alonski]

@Robdel12 I'm not sure if he is still helping out with this addon :) @karlbecker it's late for me. I'll try to answer tomorrow :)

[flexyford]

@karlbecker have you considered using the post method as described in the README to keep the dataset in-sync with the server? https://github.com/adopted-ember-addons/ember-impagination/blob/master/README.md#updating-the-state

whenItemsAddedToTopOfList(items) {
  dataset.post(items); // Adds items to beggining of dataset
}

impagination does not directly support this style of data-fetching, but I would be curious what other information you may need to support this.

---

@Alonski I'm no longer the maintainer of this addon, but I'll pop my head in when I'm @'d. Sorry if my support has been lacking in the past. I hope to provide solutions to impagination proper whenever I can.

[karlbecker]

cc'ing my teammate @tiandavis to loop him in. I haven't done much of the actual Ember development, but I guess I can at least start conversations 😊 Thanks for the feedback everyone, happy to hear and learn more about other ideas, too!

[Robdel12]

@Alonski that's true, but he still owns the underlying JS package that powers this addon :)

[Alonski]

Hmm so I guess using post put and delete you could manually keep the state correct. I would love to see an implementation of this 😃 @flexyford @Robdel12 Didn't mean to offend or anything 😄 Just didn't want to bother the both of you after you passed this addon down to the adopted ember addons org. Thanks for the help!

[Alonski]

@karlbecker Still need help with this?

[karlbecker]

Thanks for the ping @Alonski - we haven't gotten back around to looking into this for awhile.

@tiandavis , any thoughts on getting more help with this, or no need to keep this issue open anymore?

Thanks!

[karlbecker]

Closing this out - we ended up using something else for pagination.