armory
armory copied to clipboard
Consider metrics hooks for detection-based metrics on evasion
Removing from the near-term milestone
@davidslater could you provide some additional description of what this needs?
Focusing just on image classification scenarios for now, what would it look like for someone to add a detection-based defense - how can we measure the FP and TP rates of the detector? How do we integrate it into the system? Can we provide final metrics that just include the samples that were not detected?
This is more of a design problem, I think. I'm not sure the best way to implement this.