Open-EO
Support paginated job listings
Pagination of job listings is becoming an important requirement to avoid performance issues due to users with hundreds/thousands of jobs. As noted in https://github.com/Open-EO/openeo-api/issues/550, paginated is client-driven, so this needs handling in python client too.
related issues
- https://github.com/Open-EO/openeo-api/issues/550
- https://github.com/Open-EO/openeo-web-editor/issues/331
- https://github.com/Open-EO/openeo-python-driver/issues/332
- https://github.com/Open-EO/openeo-geopyspark-driver/issues/959
- https://github.com/Open-EO/openeo-aggregator/issues/164
Here's a list of things that can be paginated for your convenience:
- [ ] GET /collections
- [ ] GET /processes
- [ ] GET /jobs
- [ ] GET /process_grapg
- [ ] GET /files
- [ ] GET /services
- [ ] Logs (Services and Jobs)
- [ ] Collection Items (if implemented)
As the approach is similar for all endpoints except logs (and somewhat collection items), I directly implemented it for all endpoints in the JS client for example. Note: In Python (and R) this might have implications on the Vue Components that get rendered in Jupyter...
with https://github.com/Open-EO/openeo-python-client/commit/76bac3b7384bc944a2e9c7c81bc7e3d4d07e40a6 I already added the limit argument to connection.list_jobs() to have very minimal support for paging (just to get first page)
I see you decided to default to null, too. I thought it would be better so set a specific value, but it would be somewhat breaking so had to go for null before the JS client will go toward 3.0.0...
yes, I just went for default null to keep the existing behavior for now (and to have tests that document the existing behavior). But next step will probably be to change the default to something like 100.
reopening. #767 is not enough to close this
e.g. still to do: how to get next page of job listing?