Gym
Gym copied to clipboard
Integrate SWE-Bench Verified
Background
SWE-Bench Verified is a popular benchmark for evaluating agents on real-world software engineering tasks.
What is SWE-Bench Verified
SWE-Bench Verified contains GitHub issues from open-source projects where agents must understand the codebase, write fixes, and pass test suites. The "Verified" subset has high-quality, human-validated examples. https://www.swebench.com/
Acceptance Criteria
- [ ] Implement SWE-Bench Verified in NeMo Gym