Widen
Pagination isn't working as expected
Me and my team were facing an issue with the pagination method. What's going on is that we've tried to use style1 as a pagination method but that isn't working. It's not clear enough how to use it. We're using http://pokeapi.com/ in our tests, and we already tried the pagination_request_style in addition to ǹext_page_token_path. We already tried passing $.next as the token_path, and we're trying to figure out how to use pagination_request_style parameter Looking at the source code we've found that there's no use for pagination_request_style parameter inside the methods.
What we've tried so far is the following configurations:
Token Path with pagination_page_size
This will result in:
When trying to use pagination_responde_style with "style1" value, we can only extract 20 records:
We opened a thread on Meltano's Slack but no one was able to help us. Maybe opening this issue could open more discussions on this topic.
Look at the documentation regarding this PR: https://github.com/Widen/tap-rest-api-msdk/pull/36. There are a number of different styles you can use.
