Custom FailedRequest and NoDataMatchingRequest exceptions
For the evaluation of CAMS2-40 model results, withing the CAMS2-83 project we download 22 different model results every day.
When requesting "cams-europe-air-quality-forecasts" data that is missing, or has not yet been delivered cdsapi raises an Exception with the following message:
the request you have submitted is not valid. There is no data matching your request. Check that you have specified the correct fields and values..
Currently, our application that handles the downloading needs to check the exception message in order to find out if the data is not yet available or if there is a different kind of error.
try:
client.retrieve("cams-europe-air-quality-forecasts", request, tmp_path)
except Exception as e:
if "There is no data matching your request" in str(e):
logger.info(f"No {model} data to be found")
return
logger.error(e)
raise FailedToDownload(f"failed to download {model}") from e
As far as I can tell, the Exception we're trying to handle comes from lines 506 to 509 of apy.py
https://github.com/ecmwf/cdsapi/blob/f3b94a9bc40f8d56b0d1ac8cc8bc84765509ef05/cdsapi/api.py#L494-L509
It would be of great help if instead of raising an plain Exception cdsapi would rise a custom exception, say FailedRequest. That way we could limit the scope of the except block to handle only FailedRequest exceptions.
Even better would be if cdsapi would raise custom NoDataMatchingRequest exception when reply["error"].get("reason", "").startswith("There is no data matching your request") or similar.