alibi-detect
alibi-detect copied to clipboard
Improve error handling for URL requests in dataset fetch methods
The requests.get()
calls in the fetch methods such as fetch_nab
and fetch_cifar10c
currently don't have any error handling, potentially resulting in unclear errors when the requested url is down. For example see:
https://github.com/SeldonIO/alibi-detect/blob/436e99efb88c922dc04cec97b0b52d88bf439d65/alibi_detect/datasets.py#L369-L371
The url requests could be wrapped in try/except blocks which log an exception message and raise an error, in a similar way to the alibi datasets methods:
https://github.com/SeldonIO/alibi/blob/3a589c4de86a89fbdb956427d0d3020de05bdfda/alibi/datasets.py#L105-L110
Logic is also needed in the tests to skip tests when the URL is down, for example:
https://github.com/SeldonIO/alibi/blob/3a589c4de86a89fbdb956427d0d3020de05bdfda/alibi/tests/test_datasets.py#L13-L17
Coming back to this. The only issue with skipping the test when a URL is down is that CI won't pick up cases where URL's are permanently dead or changed. Ideally we need to skip temporarily dead URL's during nightly and PR tests, but still check these less frequently i.e. during CI for releases.
Fixed by #350.
Reopening this since we missed a utility function in alibi_detect.datasets
; the corruption_types_cifar10c
function causes a CI failure if https://storage.googleapis.com/seldon-datasets
is down (see https://github.com/SeldonIO/alibi-detect/runs/6662544833). We should add the try-except
logic added in #350 to any tests using this function.