earthaccess icon indicating copy to clipboard operation
earthaccess copied to clipboard

Published 20 hours ago • verified publisher icon nsidc

Add support for `CMR-Search-After` to avoid lost / duplicated results

Open MattF-NSIDC opened this issue 3 years ago • 1 comments

It's possible with a paging query to receive duplicate or lost results if changes to CMR are occurring between page queries. The recommended way to avoid this is currently the CMR-Search-After header ("scrolling" is deprecated): https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#search-after

MattF-NSIDC avatar Sep 28 '22 16:09 MattF-NSIDC

This is definitely something worth adding! right now we are using explicit pagination but is not optimal. Maybe we can couple this pagination with some lazy query evaluation see #120

betolink avatar Sep 28 '22 18:09 betolink

This as implemented in v0.4.7!! thanks for the report @MattF-NSIDC !!

betolink avatar Feb 23 '23 18:02 betolink