armory icon indicating copy to clipboard operation
armory copied to clipboard

Consider metrics hooks for detection-based metrics on evasion

Open davidslater opened this issue 4 years ago • 3 comments

davidslater avatar Apr 22 '20 21:04 davidslater

Removing from the near-term milestone

davidslater avatar May 07 '22 04:05 davidslater

@davidslater could you provide some additional description of what this needs?

swsuggs avatar Jun 01 '22 15:06 swsuggs

Focusing just on image classification scenarios for now, what would it look like for someone to add a detection-based defense - how can we measure the FP and TP rates of the detector? How do we integrate it into the system? Can we provide final metrics that just include the samples that were not detected?

This is more of a design problem, I think. I'm not sure the best way to implement this.

davidslater avatar Jun 01 '22 15:06 davidslater