Two canary clusters held and zero moved this week.

This week both canary clusters held, so clustersHeld is 2 and clustersMoved is 0, meaning the two routes stayed with their stored model picks and no route was rerouted. The agentic-tool-use cluster, a reasoning task where a model chooses and calls tools, ran a four-item canary that scored a mean of 0.95 against a stored quality of 0.97, give or take 0.147 for the stored 95% interval half-width, and the verdict was ok, so the stored pick is still confirmed. The classification cluster, a structured-output task where a model assigns labels to inputs, ran a four-item canary that scored a mean of 1 against a stored quality of 0.888, give or take 0.083 for the stored 95% interval half-width, and the verdict was ok, while eight items were graded in the proposals pipeline and zero new models were measured this week.

This week's frontiers

One row per kind of work. Routed pick is the model Potion currently sends that work to. Stored quality is its exam score when it was measured in full, give or take the margin. Canary is this week's small re-check: a few fresh tasks, scored the same way, to catch a model that has got worse. Held means the re-check landed inside the margin.

The two canary clusters are the weekly re-checks: agentic-tool-use in the reasoning family and classification in the structured-output family. Each canary is a small sample of four items run against the stored measurement to detect collapse, not one-point movement, so a held verdict means the mean did not fall below the stored interval's lower bound. Both clusters held this week, so neither stored pick was flagged for full re-measurement and both routes remain unchanged. The stored picks are both name withheld, so the specific model behind each route is not named in this report.

Auditions

An audition is a newly released model's first exam, on the kind of work it looks suited to. It earns a place only by beating the model already doing that work on quality or price.

No newly released models were tested this week, since the auditions list is empty and candidatesMeasured is 0. That means no fresh model entered the measured pool, so nothing new is available to challenge the stored picks or the proposals on the table.

Mixing

Mixing means using two or three cheaper models together in a particular way instead of one expensive one. We score combinations by replaying results we already have, so this costs nothing to explore. How a combination works is part of the product and is not published; what it achieves is.

No model combinations were tested this week, since the mixing list is empty. Combinations are replayed from stored item-level results and modelled conservatively, so any combination figure would understate rather than overstate the gain, but there is no combination figure to report this week.

What it means for you

A buyer paying per request gets stability this week: both routes held, so the model behind each request type did not degrade and the cost per request stays as expected. Two cost-down proposals are waiting, one for a writing cluster suggesting a 0.978 projected saving and one for classification with equal quality at 0.667, but neither is approved, so no price change is locked in. With zero new models measured, there is no cheaper or better option verified this week, so the safe move is to keep the current routes until a proposal is approved.

How the numbers are made

How these numbers are made, in plain words. We sort requests into kinds of work: sorting text into categories, pulling fields out of documents, writing code, answering from a set of documents, and so on. For each kind we keep a private exam of tasks the models have never seen. Every model sits the same exam. Code is marked by running it; other answers are marked against a reference answer. A model's quality is its average mark, and because an exam is a sample we also give a margin of error: two models whose margins overlap are called a tie. Cost is what a thousand requests would cost at the provider's public prices. A frontier is the short list of models that are the best deal at their level of quality, meaning nothing else is both better and cheaper. Each week we re-check every model on that list with a few fresh tasks to catch any that have got worse, and we give newly released models the exam for the work they look suited to.

• Every quality figure is a mean over a retrieval-hostile suite with a bootstrap 95% interval; two points whose intervals overlap are reported as tied.
• Code-generation and code-review quality is scored by executing the code; other clusters are scored by a rubric against a reference answer.
• A canary is a small weekly sample (four items) against the stored measurement; it detects collapse, not one-point movement.
• Combinations of models are replayed from stored item-level results; agreement between models is modelled conservatively, so the combination figures understate rather than overstate.

Numbers

Questions

Did any AI route change this week?

No. Both canary clusters held, so clustersMoved is 0 and neither route was rerouted. The agentic-tool-use route and the classification route both kept their stored model picks.

What is the quality of the classification model this week?

The classification canary mean was 1 over four items, against a stored quality of 0.888, give or take 0.083 for the stored 95% interval half-width. The verdict was ok, so the stored pick is still confirmed and did not collapse.

Are there any cheaper models proposed this week?

Two proposals are on the table. A writing-cluster proposal suggests a projected saving of 0.978 with the serving model scoring 0.569 against an incumbent quality of 0.706, and a classification proposal shows equal quality at 0.667 with no projected saving. Neither proposal is approved yet.

Glossary. A frontier is the short list of models that are the best deal at their level of quality: nothing else is both better and cheaper. A floor is the lowest exam score you are willing to accept. A margin (or interval) is how far the true score could sit from the measured one, because an exam is a sample. A canary is a small weekly re-check. See the docs and the evidence.