dlt
dlt copied to clipboard
RESTClient: improve paginator tests and docs
Feature description
Documentation:
- [ ] Document
SinglePagePaginator
- [ ] Write why it is helpful to declare different types during type checking compared to code execution time. To me, this decision is not obvious.
Tests:
- [ ] test_paginators.py: assert that the initial state of the paginator is as intended before we call
update_state()
. Otherwise, tests could be green ifupdate_state()
has no effect. - [ ] Consider adding a test case that the first request is without the pagination parameter and only starting from the second page do we find a
page=2
in the request params
Paginator implementation
- [ ] for better unit-testing: consider changing the signature of
update_request(self, request: Request) -> None
toupdate_request(self, request: Request) -> Request
Related issues
https://github.com/dlt-hub/dlt/issues/1325 Follow-up to https://github.com/dlt-hub/dlt/pull/1509