Azure
FEAT Distributed running of attacks
Is your feature request related to a problem? Please describe.
I want to be able to run multi-turn attacks on multiple datasets in parallel that can scale to multiple machines.
Additional context
The current multi-turn attacks like PAIR, Crescendo, TAP etc. are all running asynchronously but are not scalable to multiple machines with GPUs. I want to system that could run a large corpus of multi-turn dataset and can process the attacks simultaneously. For example, if I am running a PAIR attack strategy, then I should be able to parallelise the data and attacks that are happening simultaneously.
Part of this will be addressed with our ongoing work towards a "scanner." At first, it'll be local only, but we're planning for remote execution in the cloud eventually. This also includes using multiple machines. I haven't really given much thought towards splitting the data into multiple parallel jobs, more so splitting different orchestrators up into their own jobs. Still, it should be doable eventually. We can keep this open as a feature request in that direction, but please keep in mind that it will take a while until we get there.
In the meantime, something that may help is using a SQL database. Then you can have multiple instance running but all saving data to the same place
