Skip to content
    FreeDAIY
    Contact us

    AI Progress Tracker, SWE-bench Verified (2024–2026)

    Last updated 2026-10-09 · Updated weekly

    SWE-BENCH VERIFIED · 500 REAL GITHUB ISSUES · Y = % ISSUES RESOLVED
    ★ = SOTA OPEN-SOURCE AT LAUNCH · TAP/HOVER DOTS FOR DETAIL · TRACKING HOW FAST AI IS MOVING, AND WHAT IT MEANS FOR YOU

    Date:
    SWE-bench score:
    Type:

    ⏱ Time for OSS to catch each frontier model

    Frontier
    Claude Opus 5 (97%)
    2026-07-24
    First OSS Match
    Not yet caught
    No OSS model has matched this score
    –
    open gap
    Frontier
    Claude Mythos 5 (95.5%)
    2026-06-09
    First OSS Match
    DeepSeek V4 Pro 0813
    2026-08-13 · 96.4%
    65d
    2.1 mo
    Frontier
    Claude Fable 5 (95%)
    2026-06-09
    First OSS Match
    DeepSeek V4 Pro 0813
    2026-08-13 · 96.4%
    65d
    2.1 mo
    Frontier
    GPT-5.5 (88.7%)
    2026-04-23
    First OSS Match
    Kimi K3
    2026-07-16 · 93.4%
    84d
    2.8 mo
    Frontier
    Claude Opus 4.7 (87.6%)
    2026-04-16
    First OSS Match
    Kimi K3
    2026-07-16 · 93.4%
    91d
    3.0 mo
    Frontier
    GPT-5.3 Codex (85%)
    2026-04-10
    First OSS Match
    Kimi K3
    2026-07-16 · 93.4%
    97d
    3.2 mo
    Frontier
    Claude Mythos Preview (93.9%)
    2026-04-07
    First OSS Match
    DeepSeek V4 Pro 0813
    2026-08-13 · 96.4%
    128d
    4.2 mo
    Frontier
    GPT-5.2 (80%)
    2025-12-11
    First OSS Match
    MiniMax M2.5
    2026-02-12 · 80.2%
    63d
    2.1 mo
    Frontier
    GPT-5.1 (76.3%)
    2025-11-12
    First OSS Match
    Kimi K2.5
    2026-01-26 · 76.8%
    75d
    2.5 mo
    Frontier
    Claude Opus 4.5 (80.9%)
    2025-11-01
    First OSS Match
    GLM-5.2
    2026-06-16 · 82.8%
    227d
    7.5 mo
    Frontier
    Claude Sonnet 4.5 (77.2%)
    2025-09-29
    First OSS Match
    GLM-5
    2026-02-11 · 77.8%
    135d
    4.4 mo
    Frontier
    GPT-5 (74.9%)
    2025-08-07
    First OSS Match
    Kimi K2.5
    2026-01-26 · 76.8%
    172d
    5.7 mo
    Frontier
    Claude Opus 4.1 (74.5%)
    2025-08-05
    First OSS Match
    Kimi K2.5
    2026-01-26 · 76.8%
    174d
    5.7 mo
    Frontier
    Claude 4 Sonnet (72.7%)
    2025-05-22
    First OSS Match
    DeepSeek-V3.2
    2025-12-01 · 73.1%
    193d
    6.3 mo
    Frontier
    o3 (71.7%)
    2025-04-16
    First OSS Match
    DeepSeek-V3.2
    2025-12-01 · 73.1%
    229d
    7.5 mo
    Frontier
    Claude 3.7 Sonnet (70.3%)
    2025-02-24
    First OSS Match
    DeepSeek-V3.2
    2025-12-01 · 73.1%
    280d
    9.2 mo
    Frontier
    Claude 3.5 Sonnet v2 (49%)
    2024-10-22
    First OSS Match
    DeepSeek-R1
    2025-01-20 · 49.2%
    90d
    3.0 mo
    Frontier
    o1 (48.9%)
    2024-09-12
    First OSS Match
    DeepSeek-R1
    2025-01-20 · 49.2%
    130d
    4.3 mo
    Frontier
    GPT-4o (33.2%)
    2024-05-13
    First OSS Match
    Qwen2.5-Coder-32B
    2024-11-11 · 39%
    182d
    6.0 mo
    ▾ FULL DATA TABLE, all models, dates, scores, sources
    #ModelReleasedScoreBarNotesSource
    1GPT-6 Sol⚠2026-09-22Pending
    		⚠ SWE-bench Verified not published (OpenAI benchmark shift to DeepSWE v1.1, where Sol scores 68.8% at max effort) — below GPT-5.6 Sol's 72.7% but far cheaper, $2.74 vs $6.46 per task.techjacksolutions.com
    2GPT-6 Luna⚠2026-09-22Pending
    		⚠ SWE-bench Verified not published — lighter/cheaper tier scoring 66.6% on DeepSWE v1.1 at max effort; OpenAI claims quality comparable to Claude Opus 5/Fable 5 at medium effort for a fraction of the cost.emergent.sh
    3Claude Opus 5.5⚠2026-09-22Pending
    		⚠ SWE-bench Verified not published, as the leaderboard is being retired industry-wide; Anthropic instead reports 89.9% SWE-bench Pro (+10.7pp over Opus 5) and 93.9% SWE-bench Multilingual. Adaptive-thinking refresh, 1M context.llm-stats.com
    4Grok 4.7⚠2026-09-21Pending
    		⚠ SWE-bench Verified not published — xAI moved to DeepSWE v1.1/FrontierSWE V2/SWE-Marathon for this release (71.0% at high effort on the DeepSWE-style metric). 2.1T params; superseded Grok 4.6 as flagship.www.orcarouter.ai
    5GPT-6 Astra⚠2026-09-03Pending
    		⚠ SWE-bench Verified not published — OpenAI shifted its primary coding benchmark to DeepSWE v1.1 for this generation, as the industry-standard Verified leaderboard was effectively retired as saturated by autumn 2026. 1.05M context, April 2026 knowledge cutoff.en.wikipedia.org
    6Gemini 3.8 Flash2026-09-0280.0%
    		80.0% SWE-bench (vals.ai independent run) vs Google's own reported SWE-bench Pro of 61.6% (up from 60.4% for 3.7 Flash). Third Flash-tier release in six weeks (3.6→3.7→3.8).deepmind.google
    7Ornith-1.5-397BOSS2026-08-1986.0%
    		86.0% SWE-bench Verified (Ornith/DeepReinforce technical report) — ahead of Claude Opus 4.8 by the same measure (85.0%). MIT license, 397B MoE, 400K context, explicit think-block reasoning, downloadable weights.www.orcarouter.ai
    8DeepSeek V4 Pro 0813OSS⚠2026-08-1396.4%
    		⚠ Contested: 96.4% SWE-bench Verified (vals.ai, mini-swe-agent harness) — would be new OSS SOTA and near parity with Claude Opus 5, but DeepSeek's own technical report cites a much lower 80.6% on a different harness; treat with caution pending reconciliation. MIT license, 1.6T MoE, 1M context.codersera.com
    9Grok 4.62026-08-1295.6%
    		95.6% SWE-bench Verified (vals.ai) — last xAI flagship benchmarked on Verified before Grok 4.7 (Sep 21) shifted to DeepSWE v1.1/FrontierSWE/SWE-Marathon. Same ~500K context class as Grok 4.5.benchlm.ai
    10Qwen3.8-MaxOSS2026-08-1087.3%
    		87.3% SWE-bench Verified (vals.ai) — ahead of GPT-5.5 (82.6%) and GLM-5.2 (83.3% per the same harness), just behind Claude Opus 4.8. 2.4T MoE; open weights for Max and the companion Qwen3.8-27B landed on Hugging Face/ModelScope ~Aug 10.datanorth.ai
    11Claude Opus 52026-07-2497.0%
    		97.0% SWE-bench Verified (vals.ai, mini-swe-agent, bash-only harness) — new public SOTA overall, surpassing the restricted-access Mythos 5 (95.5%). 1M context, $5/$25 per M tokens.www.vals.ai
    12Kimi K3OSS★2026-07-1693.4%
    		93.4% SWE-bench Verified (vals.ai, mini-swe-agent harness) — new OSS SOTA, ahead of Claude Opus 4.8 (88.6%); Moonshot itself doesn't publish a Verified score, reporting DeepSWE/Terminal-Bench instead. 2.8T MoE, 104B active, 1M context; open weights July 27.www.vals.ai
    13Claude Sonnet 52026-06-3085.2%
    		85.2% SWE-bench Verified (benchlm.ai aggregate); mid-tier refresh shipped alongside Fable 5/Mythos 5's global redeployment after a brief export-control suspension.www.artificialintelligence-news.com
    14GLM-5.2OSS★2026-06-1682.8%
    		82.8% SWE-bench Verified (benchmark tracking sources, since independently measured) — new OSS SOTA; Z.AI published no Verified number at launch, citing SWE-bench Pro (62.1%) instead. Flagship 744B-total / 40B-active MoE, 1M-token context, dual reasoning modes.apidog.com
    15Kimi K2.7 CodeOSS⚠2026-06-1278.2%
    		⚠ Vendor-reported: 78.2% SWE-bench Verified (Moonshot AI); no independent third-party verification at launch. Coding-focused 1T-parameter open-weight; uses 30% fewer reasoning tokens than K2.6.huggingface.co
    16DiffusionGemma 26B-A4BOSS2026-06-1022.9%
    		22.9% SWE-bench Verified (Google DeepMind model card) — trades coding accuracy for speed. Experimental open-weight diffusion-based text LLM (26B MoE, Apache 2.0); generates 4× faster than autoregressive Gemma 4.ai.google.dev
    17Claude Fable 52026-06-0995.0%
    		95.0% SWE-bench Verified — new public SOTA, #2 on BenchLM leaderboard. First publicly-available Mythos-class model (tier above Opus 4.8); 1M context.www.anthropic.com
    18Claude Mythos 52026-06-0995.5%
    		95.5% SWE-bench Verified — #1 on BenchLM leaderboard. Restricted-access variant of Fable 5 (Project Glasswing tier); same weights, lifted safeguards for authorized cybersecurity orgs.www.anthropic.com
    19North Mini Code 1.0OSS2026-06-0967.6%
    		67.6% SWE-bench Verified pass@1, confirmed (SWE-Agent harness) — the widely repeated 80.2% figure was the pre-RLVR SFT checkpoint's pass@10, not pass@1. 30B-A3B MoE, Apache 2.0; single H100 deployable.www.spheron.network
    20Unisound U2⚠2026-06-0775.0%
    		⚠ Vendor-reported 75% SWE-bench Verified (Unisound press release). Chinese agentic foundation model capable of 100+ step autonomous workflows.www.prnewswire.com
    21Nemotron 3 UltraOSS2026-06-0471.9%
    		71.9% SWE-bench Verified (NVIDIA technical report, BenchLM); 550B-total / 55B-active hybrid Mamba-Transformer MoE optimized for long-running agents.research.nvidia.com
    22MAI-Code-1-Flash2026-06-0271.6%
    		71.6% SWE-bench Verified (Microsoft AI announcement); first in-house Microsoft coding model — 137B sparse MoE, 5B active. Launched at Build 2026; beats Claude Haiku 4.5 (66.6%).microsoft.ai
    23MAI-Thinking-12026-06-0273.5%
    		73.5% SWE-bench Verified (BenchLM); first in-house Microsoft reasoning model. 35B active / 1T total MoE, trained from scratch without OpenAI distillation.microsoft.ai
    24MiniMax M3OSS2026-06-0180.5%
    		80.5% SWE-bench Verified, now independently confirmed by vals.ai and Artificial Analysis; 59% SWE-bench Pro. Open-weight 1M-context MoE with MSA architecture, native image/video input.www.vals.ai
    25Grok Build 0.1⚠2026-05-2970.8%
    		⚠ Vendor-reported: 70.8% SWE-bench Verified (xAI announcement, own harness); no independent score yet. Dedicated coding model behind Grok Build CLI, 256K context.x.ai
    26Step 3.7 FlashOSS2026-05-2974.4%
    		74.4% SWE-bench Verified (NVIDIA NIM docs); some sources cite 76.5% on a variant flagged for contamination — SWE-bench Pro (56.3%) is the cleaner signal. StepFun 198B sparse MoE vision-language coding model, Apache 2.0.docs.api.nvidia.com
    27Claude Opus 4.82026-05-2888.6%
    		88.6% SWE-bench Verified (llm-stats.com, BenchLM); adaptive-thinking Opus refresh with 1M context window.www.anthropic.com
    28Qwen3.7-Max2026-05-2080.4%
    		80.4% SWE-bench Verified (felloai.com, DataCamp, startupfortune.com); 60.6% SWE-bench Pro (highest at launch). Closed API flagship. 1M context. Launched at Alibaba Cloud Summit Hangzhou.qwen.ai
    29Gemini 3.5 Flash2026-05-1978.0%
    		78% SWE-bench Verified (Google I/O 2026); beats Gemini 3.1 Pro on 11/15 agentic benchmarks; 4× faster than rival frontier models; 55.1% SWE-bench Pro. Closed model.blog.google
    30GPT-5.5 Instant⚠2026-05-05Pending
    		⚠ SWE-bench Verified pending — no independent score published. Lighter/faster default-tier variant of GPT-5.5; became the new ChatGPT default May 5. $1.50/$6/M tokens.techcrunch.com
    31Grok 4.32026-04-3071.4%
    		71.4% SWE-bench Verified (vals.ai); improved agentic performance over Grok 4.20; 40% price cut, $0.80/$3.20/M tokens.www.vals.ai
    32Mistral Medium 3.5OSS2026-04-2977.6%
    		77.6% SWE-bench Verified (MarkTechPost / winbuzzer); unified 128B model merging chat+reasoning+code, replaces Devstral 2 and Magistral. Modified MIT license. 256K context, $1.50/M input.huggingface.co
    33DeepSeek-V4-ProOSS★2026-04-2480.6%
    		80.6% SWE-bench Verified — new OSS SOTA, 0.4pp above MiniMax M2.5/Kimi K2.6. 1.6T param MoE, 49B active, 1M context. MIT license. $0.30/$1.20/M tokens.huggingface.co
    34DeepSeek-V4-FlashOSS2026-04-2479.0%
    		79.0% SWE-bench Verified (BenchLM); 284B-A13B MoE — 12× cheaper than V4-Pro at 1.6pp lower SWE-bench Verified. MIT license, 1M context.huggingface.co
    35GPT-5.52026-04-2388.7%
    		88.7% SWE-bench Verified — new public SOTA, beats Claude Opus 4.7 (87.6%). First fully retrained base since GPT-4.5. 1M context, $5/$30/M tokens. Codename 'Spud'.openai.com
    36Hy3-previewOSS2026-04-2374.4%
    		74.4% SWE-bench Verified (Artificial Analysis, multiple sources); 40% jump from Hy2 (53.0%). Tencent. 295B MoE / 21B active, 256K context. Tencent Hy Community License.huggingface.co
    37Qwen3.6-27BOSS2026-04-2277.2%
    		77.2% SWE-bench Verified (llm-stats.com / the-decoder.com); 27B dense model, Apache 2.0. Surpasses Qwen3.5-397B-A17B on all major coding benchmarks despite 14× fewer params.huggingface.co
    38MiMo-V2.5-ProOSS2026-04-2278.9%
    		78.9% SWE-bench Verified (artificialanalysis.ai, the-decoder.com); 57.2% SWE-bench Pro. Xiaomi. 1.02T MoE, 42B active, MIT license, 1M context.huggingface.co
    39Qwen3.6-Max-Preview2026-04-2076.7%
    		76.7% SWE-bench Verified (official). Leads SWE-bench Pro, Terminal-Bench 2.0, SkillsBench at launch (#1 on 6 benchmarks). Closed API flagship from Alibaba. 260K context.qwen.ai
    40Kimi K2.6OSS2026-04-2080.2%
    		80.2% SWE-bench Verified; 58.6% SWE-bench Pro. 1T params / 32B active MoE. 300-agent swarm, 4000-step horizon, 262K context. Modified MIT.huggingface.co
    41Claude Opus 4.72026-04-1687.6%
    		87.6% SWE-bench Verified; 64.3% SWE-bench Pro (#1 public model at launch). xhigh effort, 3x vision, /ultrareview, 1M context. $5/$25 per M tokens.www.anthropic.com
    42Qwen3.6-35B-A3BOSS2026-04-1673.4%
    		73.4% SWE-bench Verified. Apache 2.0. 35B total / 3B active MoE — runs on a laptop. Highest OSS SWE score on consumer hardware.huggingface.co
    43GPT-5.3 Codex2026-04-1085.0%
    		85.0% SWE-bench Verified — OpenAI coding-specialist Codex variant; 56.8% SWE-bench Pro (benchlm.ai)benchlm.ai
    44Muse Spark2026-04-0877.4%
    		77.4% SWE-bench Verified (The Next Web, llm-stats.com, vals.ai); 55.0% SWE-bench Pro. Meta Superintelligence Labs — first flagship closed-source Meta model. Natively multimodal.ai.meta.com
    45Claude Mythos Preview2026-04-0793.9%
    		93.9% SWE-bench Verified — most capable model ever built; NOT publicly released. Restricted to 50 orgs under Project Glasswing (cybersecurity). Triggered ASL-4 safety protocol. $25/$125 per M tokens.www.buildfastwithai.com
    46GLM-5.1OSS2026-04-0777.8%
    		77.8% SWE-bench Verified (llm-stats.com); 58.4% SWE-bench Pro — SOTA at launch. Post-training upgrade to GLM-5 for agentic coding. 744B MoE, MIT license.huggingface.co
    47Qwen3.6 Plus2026-04-0278.8%
    		78.8% SWE-bench Verified (OpenRouter benchmarks; 56.6% SWE-bench Pro); closed API flagship from Alibaba; 1M context. Tops Terminal-Bench 2.0 agentic coding at launch.qwen.ai
    48Gemma 4 26B A4B (MoE)OSS⚠2026-04-0264.0%
    		⚠ SWE-bench pending (no official Google score). Multiple third-party sources ~64% on SWE-bench Verified; 77.1% LiveCodeBench v6. Runs on 24GB GPU — 3.8B active params; 256K context; Apache 2.0.huggingface.co
    49Gemma 4 31B (Dense)OSS⚠2026-04-0264.0%
    		⚠ SWE-bench pending (no official Google score). Multiple third-party sources confirm ~64% on SWE-bench Verified. LiveCodeBench v6: 80.0%. #3 open model on Arena AI (ELO 1452); single 80GB GPU; 256K context; Apache 2.0.huggingface.co
    50MiniMax M2.7OSS2026-03-1778.0%
    		78.0% SWE-bench Verified (regression from M2.5's 80.2%; self-improvement focus over coding perf); 56.22% SWE-Pro (SOTA at launch); SWE Multilingual 76.5%. MIT license, 204K context.www.minimax.io
    51GPT-5.42026-03-0578.2%
    		GPT-5.4: 78.2% bash-only (vals.ai); 57.7% SWE-bench Prowww.vals.ai
    52Claude Sonnet 4.62026-02-1679.6%
    		79.6% SWE-bench Verified (llm-stats.com); $3/$15 per M tokenswww.anthropic.com
    53Qwen3.5-27BOSS2026-02-1672.4%
    		Dense 27B model; remarkable performance for its sizeawesomeagents.ai
    54Qwen3.5-397B-A17BOSS2026-02-1676.4%
    		Large MoE Qwen3.5; strong all-round but behind K2.5llm-stats.com
    55Claude Opus 4.62026-02-1380.8%
    		80.8% (llm-stats.com); 78.2% bash-only (vals.ai)llm-stats.com
    56Gemini 3.1 Pro2026-02-1380.6%
    		80.6% (llm-stats.com); 78.8% bash-only (vals.ai)llm-stats.com
    57MiniMax M2.5OSS★2026-02-1280.2%
    		80.2% SWE-bench Verified — highest-scoring open-weight model at launch; 230B MoE, MIT license, $0.30/$1.20 per M tokenshuggingface.co
    58GLM-5OSS★2026-02-1177.8%
    		77.8% SWE-bench Verified (llm-stats.com); OSS SOTA for 1 day before MiniMax M2.5. 744B MoE, 40B active, MIT license. Z.AI (Zhipu).huggingface.co
    59Qwen3-Coder-NextOSS2026-02-0370.6%
    		70.6% SWE-Agent / 71.3% OpenHands; 80B/3B-active MoE, 262K ctx — purpose-built agentic coding, Apache 2.0qwen.ai
    60Step-3.5-FlashOSS2026-02-0274.4%
    		74.4% SWE-bench Verified; 196B-A11B MoE, 256K context. StepFun. Apache 2.0.huggingface.co
    61Kimi K2.5OSS★2026-01-2676.8%
    		K2.5 — best open model at launch; visual agentic capabilitieswww.kimi.com
    62GLM-4.7OSS★2025-12-2273.8%
    		73.8% SWE-bench Verified — OSS SOTA at launch (+5.8pp over GLM-4.6 / DeepSeek-V3.2); 355B MoE, MIT, 200K context, preserved thinking across turns. Z.AI (Zhipu).huggingface.co
    63Gemini 3 Flash2025-12-1678.0%
    		78.0% SWE-bench Verified (llm-stats.com); Flash-level latency at Pro-grade reasoningllm-stats.com
    64GPT-5.22025-12-1180.0%
    		GPT-5.2 at 80% (llm-stats.com)llm-stats.com
    65Kimi K2 ThinkingOSS2025-12-1072.8%
    		K2 + thinking mode; best OSS pass@1 on swe-rebench Dec 2025swe-rebench.com
    66Devstral 2 (Dec 2512)OSS2025-12-0972.2%
    		72.2% SWE-bench Verified (Mistral official); 123B dense model, modified MIT license, 256K context. Ships with Mistral Vibe CLI for end-to-end code automation.mistral.ai
    67Devstral Small 2OSS2025-12-0968.0%
    		68.0% SWE-bench Verified (Mistral official); 24B compact variant, Apache 2.0 — laptop-deployable companion to Devstral 2. 28× fewer params, only 4.2pp below the 123B sibling.mistral.ai
    68DeepSeek-V3.2OSS★2025-12-0173.1%
    		73.1% — top open model through late 2025 (llm-stats.com)llm-stats.com
    69Gemini 3 Pro2025-11-1876.2%
    		Gemini 3 Pro (vals.ai)www.vals.ai
    70GPT-5.12025-11-1276.3%
    		76.3% SWE-bench Verified (llm-stats.com); configurable reasoning effortllm-stats.com
    71Claude Opus 4.52025-11-0180.9%
    		80.9% — #1 overall at launch (codesota.com / llm-stats.com)www.codesota.com
    72Claude Sonnet 4.52025-09-2977.2%
    		77.2% SWE-bench Verified (82.0% high-compute); 30-hour autonomous coding capability; #1 coding + computer-use at launch (TechCrunch, Fortune, Anthropic blog)www.anthropic.com
    73Grok Code Fast 1⚠2025-08-2670.8%
    		⚠ Contested score: xAI self-reported 70.8% (own harness); independent third-party evaluation ~57.6% — significant harness gap. Purpose-built agentic coder; 314B MoE, 256K context. $0.20/$1.50/M tokens. Available via GitHub Copilot, Cursor, Cline at launch.x.ai
    74GPT-52025-08-0774.9%
    		GPT-5 SWE-bench Verified (vals.ai)www.vals.ai
    75Claude Opus 4.12025-08-0574.5%
    		74.5% SWE-bench Verified; improved multi-file refactoring and long-context reasoning over Opus 4 (Anthropic blog, InfoQ)www.anthropic.com
    76DeepSeek-V3.1OSS2025-08-0167.0%
    		Hybrid thinking mode; strong agentic/tool-use workflowswww.bentoml.com
    77Qwen3-Coder-480BOSS★2025-07-2269.6%
    		69.6% (500-turn); 67.0% standard — rivalled Claude Sonnet 4qwenlm.github.io
    78Kimi K2OSS★2025-07-1665.8%
    		Best open model at launch; 1T-param MoE, Apache 2.0arxiv.org
    79DeepSeek-R1-0528OSS2025-05-2857.0%
    		R1 update — improved reasoning, MIT licensegithub.com
    80Claude 4 Sonnet2025-05-2272.7%
    		72.7% (llm-stats.com)llm-stats.com
    81Claude Opus 42025-05-2272.5%
    		72.5% (llm-stats.com)llm-stats.com
    82Devstral (May 2505)OSS2025-05-0146.8%
    		Mistral purpose-built SWE coding agent; first of its kindmistral.ai
    83Qwen3-235B-A22BOSS★2025-04-2959.0%
    		First open model above 55% on SWE-bench Verifiedqwenlm.github.io
    84o32025-04-1671.7%
    		First OpenAI model above 70% SWE-bench Verifiedopenai.com
    85Llama 4 MaverickOSS2025-04-0532.0%
    		400B MoE; strong multimodal, weaker SWE vs Qwen/DeepSeekai.meta.com
    86Gemini 2.5 Pro2025-03-2563.8%
    		63.8% with custom agent setup (Google blog)blog.google
    87DeepSeek-V3-0324OSS2025-03-2446.0%
    		V3 update with improved reasoning; MIT licenseen.wikipedia.org
    88Gemma 3 27BOSS2025-03-1217.0%
    		Strong general model, not coding-focused; low SWE scoredeepmind.google
    89GPT-4.52025-02-2738.0%
    		Less reasoning-focused than o1; lower SWE scoreopenai.com
    90Claude 3.7 Sonnet2025-02-2470.3%
    		70.3% custom harness; 63.7% bare agent (Anthropic blog)www.anthropic.com
    91Gemini 2.0 Flash2025-01-2135.0%
    		SWE-bench Verified approximateblog.google
    92DeepSeek-R1OSS★2025-01-2049.2%
    		First open reasoning model; matched OpenAI o1 on math/codegithub.com
    93DeepSeek-V3OSS★2024-12-2642.0%
    		GPT-4o parity at ~$6M training cost — shocked the industryen.wikipedia.org
    94Qwen2.5-Coder-32BOSS★2024-11-1139.0%
    		Best OSS coder at launch; matched GPT-4o on codingqwenlm.github.io
    95Claude 3.5 Sonnet v22024-10-2249.0%
    		Oct 2024 update; 49% SWE-bench Verifiedwww.codesota.com
    96Qwen2.5-72BOSS2024-09-1918.0%
    		General model; moderate SWE-bench via Agentless scaffoldqwenlm.github.io
    97o12024-09-1248.9%
    		First OpenAI chain-of-thought reasoning modelopenai.com
    98Llama 3.1 70BOSS★2024-07-2311.0%
    		Best open-source non-specialist at the timeai.meta.com
    99Claude 3.5 Sonnet2024-06-2015.0%
    		Initial release; ~15% via early agent scaffoldswww.anthropic.com
    100GPT-4o2024-05-1333.2%
    		OpenAI self-reported best result on SWE-bench Verifiedopenai.com
    101Claude 3 Opus2024-03-048.0%
    		SWE-bench Verified via RAG scaffoldwww.anthropic.com
    102Gemini 1.5 Pro2024-02-156.0%
    		Approximate via early agent evalsblog.google
    103Gemini 1.0 Pro2023-12-061.5%
    		Estimated — very limited coding capability at launchblog.google
    104SWE-Llama 13BOSS★2023-10-103.5%
    		Princeton baseline — first open model on SWE-benchwww.swebench.com
    105Claude 22023-07-114.8%
    		SWE-bench original via RAG; pre-Verified periodwww.anthropic.com
    106GPT-42023-03-1416.0%
    		Best scaffold (CodeR) on SWE-bench Verified; raw model ~2–4%openai.com
    107GPT-3.5 (ChatGPT)2022-11-303.8%
    		SWE-bench original (not Verified); earliest public LLM entryopenai.com