JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks

[ramimac]

https://arxiv.org/pdf/2312.10766

we propose JailGuard, a universal detection framework for jailbreaking and hijacking attacks across LLMs and MLLMs. JailGuard operates on the principle that attacks are inherently less robust than benign ones, regardless of method or modality. Specifically, JailGuard mutates untrusted inputs to generate variants and leverages discrepancy of the variants’ responses on the model to distinguish attack samples from benign samples