astronaut icon indicating copy to clipboard operation
astronaut copied to clipboard

fix pageToken handling

Open drassi opened this issue 2 years ago • 0 comments

Encountered a small bug in page token handling when looking at the crawler a few days ago. For paged queries, 'nextPageToken' is sent as a param to search API, but actually 'pageToken' is the correct input param name (even though nextPageToken/prevPageToken are what's returned). Ref https://developers.google.com/youtube/v3/docs/search/list#pageToken

I found this was causing paging to not work correctly - the first queries were being duplicated without proceeding onto the next page. This was mainly triggered with query efficiency modifications I had to make because my API limits were very low, but I believe it could be triggered under certain conditions with existing query planning if many videos under min duration are returned.

Thanks!

drassi avatar Jun 15 '23 18:06 drassi