What is the best model for code review?
As of September 18, 2026, the cheapest live-measured option within 90% of top quality on code review is gemma-4-31b-it at $0.0458 per 1,000 requests — 0.932 measured quality vs 0.967 at the top, with a 7× price spread across the measured frontier (frontier v9).
re-measured weekly · how these numbers are made
Code review asks a model to find what is wrong with working-looking code — a fundamentally harder measurement problem than generation, because a review is only as good as the defects it catches. The measured suite scores reviews against known injected defects with executable checks wherever the fix can be run.
The measured frontier
Every row is a live measurement on Potion's held-out code review suite — same items, same scoring, per option. Names that are part of the product are withheld; their numbers are not.
| option | vendor | measured quality | $ / 1K requests | p95 latency |
|---|---|---|---|---|
| mistral-small-3.2-24b-instruct | mistralai | 0.507 | $0.0284 | 5731 ms |
| Potion's routed pick (name withheld) | — | 0.858 | $0.0364 | 4454 ms |
| gemma-4-31b-it | 0.932 | $0.0458 | 11296 ms | |
| gemini-2.5-flash-lite | 0.771 | $0.0647 | 2338 ms | |
| gpt-5.4-nano | openai | 0.896 | $0.1031 | 3336 ms |
| gpt-4.1-mini | openai | 0.967 | $0.1081 | 2753 ms |
| gemini-3.1-flash-lite | 0.936 | $0.1850 | 2210 ms | |
| gemini-2.5-flash | 0.732 | $0.2002 | 2185 ms | |
| mistral-medium-3.1 | mistralai | 0.750 | $0.2296 | 2176 ms |
| gemini-3.5-flash-lite | 0.871 | $0.3160 | 1809 ms |
frontier v9 · measured 2026-09-18 · live provider calls only — simulated evidence never appears on this page
Head to head
gemma-4-31b-it vs mistral-small-3.2-24b-instructgemini-2.5-flash-lite vs mistral-small-3.2-24b-instructgpt-5.4-nano vs mistral-small-3.2-24b-instructgpt-4.1-mini vs mistral-small-3.2-24b-instructgemini-3.1-flash-lite vs mistral-small-3.2-24b-instructgemini-2.5-flash vs mistral-small-3.2-24b-instructmistral-medium-3.1 vs mistral-small-3.2-24b-instructgemini-3.5-flash-lite vs mistral-small-3.2-24b-instructgemini-2.5-flash-lite vs gemma-4-31b-itgemma-4-31b-it vs gpt-5.4-nanogemma-4-31b-it vs gpt-4.1-minigemini-3.1-flash-lite vs gemma-4-31b-itgemini-2.5-flash vs gemma-4-31b-itgemma-4-31b-it vs mistral-medium-3.1gemini-3.5-flash-lite vs gemma-4-31b-itgemini-2.5-flash-lite vs gpt-5.4-nanogemini-2.5-flash-lite vs gpt-4.1-minigemini-2.5-flash-lite vs gemini-3.1-flash-litegemini-2.5-flash vs gemini-2.5-flash-litegemini-2.5-flash-lite vs mistral-medium-3.1gemini-2.5-flash-lite vs gemini-3.5-flash-litegpt-4.1-mini vs gpt-5.4-nanogemini-3.1-flash-lite vs gpt-5.4-nanogemini-2.5-flash vs gpt-5.4-nanogpt-5.4-nano vs mistral-medium-3.1gemini-3.5-flash-lite vs gpt-5.4-nanogemini-3.1-flash-lite vs gpt-4.1-minigemini-2.5-flash vs gpt-4.1-minigpt-4.1-mini vs mistral-medium-3.1gemini-3.5-flash-lite vs gpt-4.1-minigemini-2.5-flash vs gemini-3.1-flash-litegemini-3.1-flash-lite vs mistral-medium-3.1gemini-3.1-flash-lite vs gemini-3.5-flash-litegemini-2.5-flash vs mistral-medium-3.1gemini-2.5-flash vs gemini-3.5-flash-litegemini-3.5-flash-lite vs mistral-medium-3.1
Questions
How is review quality measured?
Against known defects: items carry deliberately introduced bugs, and a review scores on whether the correction actually repairs them — executed where possible, never style-judged.
Is the cheapest reviewer good enough for CI?
The measured gap on this workload is narrower than on classification — read the interval, not just the point. For gating merges, the honest configuration is a quality floor set from your own traffic, not a benchmark number.
Can review be combined with generation?
Potion measured multi-model combinations extensively and published the negative: for measured task shapes at current prices, combinations are dominated by the best single model. The write-up is in the research archive.
Potion routes each request to the cheapest option measured at your quality bar — with a receipt on every answer and this page's evidence behind every pick. Get an API key or read the docs.