planet-client-python icon indicating copy to clipboard operation
planet-client-python copied to clipboard

Too-large search filter geometry causes opaque error and futile retries

Open sgillies opened this issue 3 years ago • 3 comments

If I search with a big ole 9MB blob of GeoJSON (attached, 1.2 MB zipped) the search appears to hang. If I turn up the logging verbosity, I see that we're in a retry loop that doesn't break.

...
2022-10-07 16:01:46,708 - planet.http - DEBUG - Worker released.
2022-10-07 16:01:46,708 - planet.http - INFO - Try 2
2022-10-07 16:01:46,708 - planet.http - INFO - Retrying: caught <class 'httpx.ReadError'>: 
2022-10-07 16:01:46,708 - planet.http - INFO - Retrying: sleeping 4.786s
...

I think that the GeoJSON may be over the size limit and that the retries cannot succeed. We're losing some signal from the API server. There's no server response in the log.

To reproduce, try this with the attached GeoJSON.

planet data filter --geom rmnp.geojson --date-range acquired gte 2022-10-06 | planet --verbosity DEBUG data search PSScene

rmnp.zip

sgillies avatar Oct 07 '22 22:10 sgillies

I just re-ran this and yep definitely it's triggering a ReadError every time. We have included ReadError in the list of errors that are retried, and the retry count is 5, with an increasing delay between each retry. Due to the long delay it may seem that the loop doesn't break, though it does with time.

There is definitely an opportunity here to treat repeating ReadErrors differently from timeouts. Not sure of a way to tell the user that the repeated ReadError is due to the geojson size tho.

jreiberkyle avatar Mar 21 '23 20:03 jreiberkyle