Stargate
Stargate copied to clipboard
Versatility of pagination strategies
Currently, we support 2 strategies for pagination of collections:
- Do not paginate at all
- Page-based pagination
In order to provide more versatility in the pagination strategies a user can use, we can also support:
- KeySet-based pagination
- Cursor-based pagination
A pagination policy now defines:
- Request parameters controlling the pagination
- Hypermedia affordances to provide, both as links or as headers in the response
- Support for using pagination data in query evaluation
Page-based pagination
This is the simplest and most common form of paging. The set is divided into pages.
- Use
start
andlimit
query parameters to control pagination - Provide
first
,last
,next
, andprevious
hypermedia affordances. If needed can provide affordances for jumping to specific pages.
When using SQL in the persistence layer, It can do a query using LIMIT and OFFSET.
Pros:
- You can jump to any particular page
- It allows sending parallel requests with different pages.
- Stateless on the server-side
Cons:
- Bad performance for large
OFFSET
in SQL. - It can return repeated or missing data if anything is added/deleted while paginating.
KeySet-based pagination
The API provides a key param that acts as a delimiter of the page. This key param should be the same key of the set sort order. For example, it can be an index (since_id
), or timestamp (since_created_at
, since_updated_at
), etc.
The first request doesn’t contain the delimiter param. The response to this request will contain the value of the key for the last element of the set, so it can be used for subsequent queries.
- Use
limit
and an API-defined query parameter to control pagination. - Provide
next
hypermedia affordance.
When using SQL in the persistence layer, it will end up doing a query including a WHERE
and ORDER BY
using the key param, and LIMIT
for the limit.
Pros:
- The SQL query can be more efficient than using
OFFSET
- New records inserted on previous pages won't cause duplicated elements.
Cons:
- It's tied to the sort order.
- There is no way to jump to a specific page. We need to iterate over all the previous pages.
- Missing items if they are added to the previous pages
Cursor-based pagination
Given a set, a cursor will be a piece of data that contains a pointer to an element and the info to get the next/previous elements. The server should return the cursor pointing to the next page in each request. In most cases, the cursor is opaque, so users cannot manipulate it.
The SQL query will depend on the implementation, but it will be similar to the query generated by the KeySet-based Pagination method using a WHERE condition.
Clients should not store the cursor on their side. Google API Documentation suggests adding an expiration date to the token and expiring cursors sent in requests.
- Use
limit
andafter/before
to control pagination. - Provide
next
and/orprevious
hypermedia affordances.
Pros:
- If the cursor is opaque, the implementation underneath can change without having to introduce an API change.
- In SQL, for most of the cases, it is much faster than using page since it won’t use OFFSET in the database.
- There is no issue when a record is deleted as opposed to Page-based Pagination
Cons:
- There is no way to skip pages.
- It doesn’t allow sending parallel requests for different batches.
- The implementation is more complex than LIMIT/OFFSET.
- Hard to debug. Given a request, you have to unencode it to see what’s doing.
- Missing items if they are added to the previous pages
Reference
See here some examples of real usage