What is the best model for code generation?
As of August 24, 2026, the cheapest live-measured option within 90% of top quality on code generation is Potion's routed pick (name withheld) at $0.0232 per 1,000 requests — 0.980 measured quality vs 1.000 at the top, with a 296× price spread across the measured frontier (frontier v5).
re-measured weekly · how these numbers are made
Code generation is the easiest workload to measure honestly — generated code either passes executable tests or it does not — and the hardest to generalize about, because "coding" spans one-shot functions to repository-scale agents. The frontier below is measured by executing every answer against held-out test suites: fractional credit per test, no judge, references gated on passing their own tests.
The measured frontier
Every row is a live measurement on Potion's held-out code generation suite — same items, same scoring, per option. Names that are part of the product are withheld; their numbers are not.
| option | vendor | measured quality | $ / 1K requests | p95 latency |
|---|---|---|---|---|
| Potion's routed pick (name withheld) | — | 0.980 | $0.0232 | 15351 ms |
| gpt-4.1-mini | openai | 0.988 | $0.2520 | 4383 ms |
| gemini-2.5-flash | 0.994 | $0.5537 | 2372 ms | |
| grok-4.6 | x-ai | 1.000 | $6.85 | 46055 ms |
frontier v5 · measured 2026-08-24 · live provider calls only — simulated evidence never appears on this page
Head to head
gemini-2.5-flash vs gpt-4.1-minigpt-4.1-mini vs grok-4.6gemini-2.5-flash vs grok-4.6
Questions
How is code quality scored?
By execution: generated code runs in a sandbox against multiple tests per item, with fractional credit. A plausible-looking answer that fails its tests scores what it earned.
Does the ranking hold for large codebases and agents?
Not automatically. These suites measure self-contained generation; repository-scale and multi-step work are measured separately (see the journey suite in the methodology). A model can excel here and lag there.
Why do saturated scores get flagged?
When multiple models ace a suite repeatedly, the suite has stopped discriminating — a saturation alarm triggers hardening rather than letting a perfect score masquerade as proof of superiority. A 42/42 is reported as a lower bound, not certainty.
Potion routes each request to the cheapest option measured at your quality bar — with a receipt on every answer and this page's evidence behind every pick. Get an API key or read the docs.