What is the best model for code review?
As of August 24, 2026, the cheapest live-measured option within 90% of top quality on code review is gpt-4.1-mini at $0.4532 per 1,000 requests — 0.909 measured quality vs 0.972 at the top, with a 7× price spread across the measured frontier (frontier v4).
re-measured weekly · how these numbers are made
Code review asks a model to find what is wrong with working-looking code — a fundamentally harder measurement problem than generation, because a review is only as good as the defects it catches. The measured suite scores reviews against known injected defects with executable checks wherever the fix can be run.
The measured frontier
Every row is a live measurement on Potion's held-out code review suite — same items, same scoring, per option. Names that are part of the product are withheld; their numbers are not.
| option | vendor | measured quality | $ / 1K requests | p95 latency |
|---|---|---|---|---|
| Potion's routed pick (name withheld) | — | 0.817 | $0.0528 | 19562 ms |
| deepseek-v4-flash-0731 | deepseek | 0.869 | $0.1794 | 41891 ms |
| gpt-4.1-mini | openai | 0.909 | $0.4532 | 5100 ms |
| gemini-3.7-flash | 0.931 | $1.47 | 12159 ms | |
| kat-coder-pro-v2.5 | kwaipilot | 0.959 | $1.77 | 9185 ms |
| gpt-4.1 | openai | 0.972 | $3.07 | 4656 ms |
frontier v4 · measured 2026-08-24 · live provider calls only — simulated evidence never appears on this page
Head to head
deepseek-v4-flash-0731 vs gpt-4.1-minideepseek-v4-flash-0731 vs gemini-3.7-flashdeepseek-v4-flash-0731 vs kat-coder-pro-v2.5deepseek-v4-flash-0731 vs gpt-4.1gemini-3.7-flash vs gpt-4.1-minigpt-4.1-mini vs kat-coder-pro-v2.5gpt-4.1 vs gpt-4.1-minigemini-3.7-flash vs kat-coder-pro-v2.5gemini-3.7-flash vs gpt-4.1gpt-4.1 vs kat-coder-pro-v2.5
Questions
How is review quality measured?
Against known defects: items carry deliberately introduced bugs, and a review scores on whether the correction actually repairs them — executed where possible, never style-judged.
Is the cheapest reviewer good enough for CI?
The measured gap on this workload is narrower than on classification — read the interval, not just the point. For gating merges, the honest configuration is a quality floor set from your own traffic, not a benchmark number.
Can review be combined with generation?
Potion measured multi-model combinations extensively and published the negative: for measured task shapes at current prices, combinations are dominated by the best single model. The write-up is in the research archive.
Potion routes each request to the cheapest option measured at your quality bar — with a receipt on every answer and this page's evidence behind every pick. Get an API key or read the docs.