rstac icon indicating copy to clipboard operation
rstac copied to clipboard

Paging for collections?

Open ateucher opened this issue 1 year ago • 1 comments

Describe the requested improvement There are items_next() and items_fetch() methods for doc_items objects. The STAC API supports pagination for collections in the same way as for items. It would be great to add _next() and _fetch() methods for collections as well. We could add doc_collections methods to items_next() and items_fetch() generics, or write collections_next() and collections_fetch() generics/methods for doc_collections objects.

Associated rstac API function In practice, the implementation would look something like this:

stac("https://cmr.earthdata.nasa.gov/stac/LPCLOUD") |> 
    collections() |> 
    get_request() |> 
    collections_next()

Additional context The NASA STAC API has many collections endpoints using this paging functionality: https://cmr.earthdata.nasa.gov/stac/LPCLOUD.

I would be happy to draft a pull request if you think this is in scope.

ateucher avatar Apr 08 '24 21:04 ateucher

@ateucher This would be fantastic! Python's pystac_client handles pagination automatically, and it would be fantastic to have it in rstac too

robbibt avatar May 08 '24 04:05 robbibt

Thank you for your suggestion.

We have implemented the collections_next() and collections_fetch() methods.

These changes are in the b-1.0.1 branch and they will be released soon on CRAN. In the meantime, you can install the development version from GitHub to use these new features right away. Here’s how you can do it using the remotes package:

# Install the remotes package if you haven't already
install.packages("remotes")

# Install the development version of rstac from GitHub
remotes::install_github("brazil-data-cube/[email protected]")

These functions are similar to the existing items_next() and items_fetch() methods.

Here’s an example of how to use them:

library("rstac")

# Example usage
stac("https://cmr.earthdata.nasa.gov/stac/LPCLOUD") |>
  collections() |>
  get_request() |>
  collections_fetch()

We would greatly appreciate any feedback you have. If you encounter any issues or have further suggestions, please let us know.

Best regards, Rolf

rolfsimoes avatar Jun 14 '24 18:06 rolfsimoes

Oh that's fantastic! Apologies I didn't get to this, very happy to see it!

ateucher avatar Jun 14 '24 21:06 ateucher