www-project-machine-learning-security-top-10
www-project-machine-learning-security-top-10 copied to clipboard
OWASP
AI² Framework – Adversarial Testing Methodology for High-Risk AI
Submission Title:
AI² Framework – Adversarial Testing Methodology for High-Risk AI
Type:
Methodology / Whitepaper
Description:
The AI² Framework introduces a structured adversarial testing methodology specifically designed for high-risk AI systems, including generative and agentic AI. It enables red teams and offensive security units to assess the unique threats these systems pose using a mapped structure of risks → tactics → techniques. The framework is aligned with the NIST AI RMF (100-1 and 600-1), MITRE ATLAS, and over 30 industry sources.
Key contributions:
- Over 100 AI risks, categorized by impact area and testing viability
- Red team–ready RTT mappings for adversarial validation
- Agentic AI-specific risk test methods
- Tooling guides and domain-specific targeting logic
- Threat modeling approach for both models and integrated applications
Relevant OWASP Top 10 ML Risks Addressed:
- Model Theft
- Prompt Injection
- Poisoning (Data + Model)
- Overreliance
- Hallucination / Fabrication
- Emergent Goals
- Agent Autonomy Risks
- Input Manipulation
- Inference Attacks
- Supply Chain Compromise
Link to Full Framework:
📄 https://doi.org/10.5281/zenodo.15236593
Optional Collaboration Offer:
Happy to support alignment of the AI² Framework with OWASP ML Security formatting or contribute to deeper integration. Open to feedback, mapping, and collaboration.
Author:
Brandon Lee
[email protected]
