sonnet vs qwen3-235b-a22b-2507 on summarization: 0.4 quality points, 28× price.

Both measured on the same held-out summarization suite: sonnet at 0.824 and $3.31, qwen3-235b-a22b-2507 at 0.820 and $0.1167 per thousand requests. Both figures come from the same held-out summarization suite, 50 scored items, measured the same way — so the comparison is like for like.

Both were run against the same held-out items on the same day, so the gap is a property of the models rather than of the test. A 28× price difference at this quality distance is the whole decision.

Someone is paying the difference. If that is you, the question worth asking is which of these two numbers your workload actually needs — and this is the measurement that answers it.

3 measurement cycles ran in the last 24 hours, 3 newly listed models were measured, no recipe reached a frontier.

The current numbers live on the measured answers; the method is public.