Evaluate against the SWE-bench benchmark

[kripper]

Duplicates

• [X] I have searched the existing issues

Summary 💡

Evaluate against the SWE-bench benchmark: https://github.com/princeton-nlp/SWE-bench

Examples 🌈

No response

Motivation 🔦

Compare with and learn from other similar open source projects.

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[ntindle]

Unstale