python-for-data-and-media-communication-gitbook icon indicating copy to clipboard operation
python-for-data-and-media-communication-gitbook copied to clipboard

Pagination using HTTP header information (GitHub)

Open hupili opened this issue 6 years ago • 4 comments

GitHub API has a special design. It returns the pagination information via HTTP header. Sometimes, one can not directly construct page skip URLs but one can continuously to "scroll back".

Example API: https://api.github.com/users/hupili/events/public?per_page=1

screenshot 2018-12-01 at 2 02 19 am

Those rel=next and rel=last URLs can be called.

hupili avatar Nov 30 '18 18:11 hupili

Typical fail example:

https://api.github.com/users/hupili/events/public?page=400&per_page=1

screenshot 2018-12-01 at 2 05 07 am

hupili avatar Nov 30 '18 18:11 hupili

try to understand and figure out the api, but still cannot get more than 300, may need your further instruction, Thanks --> https://github.com/hupili/github-activies/blob/master/github_activity.ipynb

ChicoXYC avatar Dec 01 '18 16:12 ChicoXYC

what is the error when you reach 300?

p.s. above URL is invalid.

hupili avatar Dec 03 '18 15:12 hupili

example: https://api.github.com/users/hupili/events/public?per_page=10&page=10

Link: <https://api.github.com/user/1227160/events/public?per_page=10&page=9>; rel="prev", <https://api.github.com/user/1227160/events/public?per_page=10&page=11>; rel="next", <https://api.github.com/user/1227160/events/public?per_page=10&page=30>; rel="last", <https://api.github.com/user/1227160/events/public?per_page=10&page=1>; rel="first"

hupili avatar Dec 04 '18 09:12 hupili