gemini-2.5-flash-lite vs mistral-small-3.2-24b-instruct for summarization
As of September 19, 2026, mistral-small-3.2-24b-instruct measures 0.766 on summarization vs 0.732 for gemini-2.5-flash-lite (+0.034), and it is also the cheaper option ($0.0713 vs $0.0815 per 1K requests).
same suite, same items, same scoring — frontier v6, measured 2026-09-19 · method
| model | vendor | measured quality | $ / 1K requests | p95 latency |
|---|---|---|---|---|
| gemini-2.5-flash-lite | 0.732 | $0.0815 | 914 ms | |
| mistral-small-3.2-24b-instruct | mistralai | 0.766 | $0.0713 | 10078 ms |
Summarization quality is subjective at the margins, which makes it exactly the workload where judge-based scores need calibration receipts. The measured suite anchors judging against references and calibrates judges against tasks with deterministic truth, publishing the correlation rather than asking for trust.
These two are part of a larger measured frontier — what is the best model for summarization shows every measured option for this workload, and other workloads rank these models differently: a model that wins here can lose on another kind of work, which is the whole argument for routing per workload rather than picking one model for everything.
Potion routes each request to the cheapest option measured at your quality bar — including picks this public page does not name. Get an API key or read the docs.