benchmarks / swe-bench-pro-hard
SWE-bench Pro V2 HARD
pass@1. Status active. Contamination risk med. Maintainer Scale Labs. Source.
| Model | Score | Date | Kind |
|---|---|---|---|
| Claude Opus 5 | 98 | 2026-10-12 | snapshot |
| Claude Fable 5.1 | 92.2 | 2026-10-12 | snapshot |
| GPT-6 Astra | 90.2 | 2026-10-12 | snapshot |
| Kimi K3 | 88.2 | 2026-10-12 | snapshot |
| Claude Sonnet 5 | 88.2 | 2026-10-12 | snapshot |
| GPT-5.6 Terra | 86.3 | 2026-10-12 | snapshot |
| GLM 5.3 | 84.3 | 2026-10-12 | snapshot |
| GPT-5.6 Sol | 82.4 | 2026-10-12 | snapshot |
| Claude Sonnet 5.5 | 70.59 | 2026-10-12 | snapshot |
| Gemini 3.8 Flash | 58.8 | 2026-10-12 | snapshot |
| Claude Haiku 5.5 | 52.94 | 2026-10-12 | snapshot |
| Claude Haiku 4.5 | 25.5 | 2026-10-12 | snapshot |