cdsapi icon indicating copy to clipboard operation
cdsapi copied to clipboard

Custom FailedRequest and NoDataMatchingRequest exceptions

Open avaldebe opened this issue 2 years ago • 0 comments

For the evaluation of CAMS2-40 model results, withing the CAMS2-83 project we download 22 different model results every day.

When requesting "cams-europe-air-quality-forecasts" data that is missing, or has not yet been delivered cdsapi raises an Exception with the following message:

the request you have submitted is not valid. There is no data matching your request. Check that you have specified the correct fields and values..

Currently, our application that handles the downloading needs to check the exception message in order to find out if the data is not yet available or if there is a different kind of error.

    try:
        client.retrieve("cams-europe-air-quality-forecasts", request, tmp_path)
    except Exception as e:
        if "There is no data matching your request" in str(e):
            logger.info(f"No {model} data to be found")
            return
        logger.error(e)
        raise FailedToDownload(f"failed to download {model}") from e

As far as I can tell, the Exception we're trying to handle comes from lines 506 to 509 of apy.py

https://github.com/ecmwf/cdsapi/blob/f3b94a9bc40f8d56b0d1ac8cc8bc84765509ef05/cdsapi/api.py#L494-L509

It would be of great help if instead of raising an plain Exception cdsapi would rise a custom exception, say FailedRequest. That way we could limit the scope of the except block to handle only FailedRequest exceptions.

Even better would be if cdsapi would raise custom NoDataMatchingRequest exception when reply["error"].get("reason", "").startswith("There is no data matching your request") or similar.

avaldebe avatar Jun 09 '22 12:06 avaldebe