Benchmarks
Quality signals
Benchmarks are imperfect, but useful when paired with price and usage. This page keeps them beside the market data instead of pretending one score tells the whole story.
| Model | Intelligence | Coding | Agentic | Input /1M | Output /1M |
|---|---|---|---|---|---|
| GPT-6 Astra | 74 | 71 | 69 | $4 | $20 |
| Claude Sonnet 5 | 72 | 76 | 73 | $3 | $15 |
| Gemini 3 Pro | 70 | 66 | 65 | $1.25 | $10 |
OpenRouter runs
Gpqa Diamond
OpenRouter runs
Tau Bench Verified Airline
Published benchmark scores per model, collected so you can compare without visiting six leaderboards.
About this data
How much should I trust a benchmark score?
Less than you would like. Benchmarks are public, which means they end up in training data, and providers choose which ones to report. Use them to rule models out, not to pick a winner — and always test shortlisted models on your own task.
Why do scores differ from the provider’s own announcement?
Different harnesses, prompts, temperature settings and answer-extraction rules produce different numbers from the same model. A vendor also reports its best configuration. Compare scores within one column, never across sources.
Does a higher score mean a better model for me?
Only if the benchmark resembles your work. A model that leads on competition mathematics may be mediocre at following your formatting instructions, and that is the thing that will actually break your product.