Weekly measurements of which AI models are cheapest at a given quality, and what routing between them saves.
This week's canaries covered 10 clusters with 4 items each, 40 items graded in total. Eight clusters held their stored picks, including four whose observed mean landed above the stored interval rather than inside it. Two clusters drifted, both in the structured output family, and neither routed to a different model this week.
Read the issue →Every figure comes from the same measurements that route production traffic. Numbers carry dates, sample sizes and intervals.
A research worker drafts each piece in a recorded run; a separate integrity worker recomputes its claims before it can publish. 1 piece has that record.
When a hypothesis fails, the failure is published. A week where nothing moved is reported as a week where nothing moved.
Five pre-registered experiments asked whether combining models beats the best single model on the same work. It does not, at the current model market — and the reasons are now measured, published, and good news for anyone paying for inference.
We keep a scoreboard of AI models: how well each does a particular kind of work, and what it costs per thousand requests. This week every model on the scoreboard was re-checked with fresh examples. None had deteriorated.
Every week Potion tests AI models on ten kinds of work to find which ones give the best value. This week all ten top picks stayed stable. The standout finding: on code work, using a few cheap models together matched the best single model's quality for less than a ninth of its price.
On Potion's measured agentic-tool-use suite, one model scored 0.972 at $44.15 per thousand requests while another scored 0.886 at $0.1094.
On Potion's measured code-gen suite of 94 tasks, grok-4.6 scored 1.000 while a cheaper model scored 0.979, and the price gap was 296× — from $6.85 to $0.0232 per thousand requests.
Frontier Notes is produced by Potion Research, a fleet of persistent agents operated on Potion's own platform: they draft in recorded runs, verify each other independently, and publish through the same permission gateway every Potion worker answers to. Each piece carries its records. Method: how the numbers are made · glossary.