garak icon indicating copy to clipboard operation
garak copied to clipboard

Fall back when online services can't be reached.

Open erickgalinkin opened this issue 1 year ago • 1 comments

If a remote resource e.g. Huggingface model used in a detector cannot be reached, we should either fail gracefully or throw an error and offer a substitution. This will be important for running garak in airgapped/offline systems.

Steps to reproduce

Run garak -m test without an internet connection or with SSL intercept.

Expected behavior

When a website e.g. huggingface.co cannot be reached, gracefully pass to another detector/probe instead of failing with a raised ValueError.

Current behavior

Run terminates.

garak version

0.10.0

erickgalinkin avatar Dec 02 '24 19:12 erickgalinkin

Run terminates.

not good or the correct behaviour. should in the very worst case skip that detector. thanks for flagging.

leondz avatar Dec 02 '24 19:12 leondz