Subscribe
AI Models

Gemini 4 Carbon tops MathArena — at 15x the cost of GPT-6.1 Sol

Monday, November 16, 2026 · 5 min read

Gemini 4 Carbon took first place on MathArena at 85.4% expected performance, ahead of Claude Fable 5.2 and GPT-6.1 Sol — but one run costs about $15.70 per problem, roughly 15 times Sol's $1.07.

Mixed$GOOGL
Gemini 4 Carbon tops MathArena — at 15x the cost of GPT-6.1 Sol

The News

MathArena added results for two new models on November 13, and Google's Gemini 4 Carbon took first place, scoring 85.4% expected performance. That put it ahead of Anthropic's Claude Fable 5.2 at 82.8% and the previous leader, OpenAI's GPT-6.1 Sol, at 80.6%. It is the first time in a while that a model other than OpenAI's has led this benchmark, which is why the result is being talked about as a shift in the leaderboard. The benchmark data is from November 13 and the report of the results came on November 14.

The catch is the cost. One run costs about $15.70 per problem on Carbon, compared with about $1.07 on Sol — roughly 15 times as much. Fable 5.2 is even more expensive per problem, at about $40.20. A benchmark measures how well a model can do when cost is no object, but in the real world cost decides what actually gets used. A model that wins on accuracy while costing 15 times more is a very different product from one that wins on both.

There are also reasons to treat the headline score with care. The confidence intervals — the range within which the true score probably sits — overlap between the models, so the gap may be smaller than it looks. Part of Carbon's score is statistically estimated rather than directly measured: because of cost, MathArena ran Carbon only on the ArXivLean evaluation and estimated the rest, according to The Decoder. And its most common failure mode was exceeding the time limit: The Decoder reports that in one case it ran a script for over 11 hours. Google's shares closed at $295.77 on Friday, November 13, down 0.98%.

Carbon took first place — but one run costs about $15.70 per problem, roughly 15 times GPT-6.1 Sol's $1.07.

Why It Matters

A change at the top of a respected benchmark matters because these leaderboards are how the industry keeps score in the model race. OpenAI's models have led MathArena for a while, so a Google model taking first place breaks a pattern and signals that Google's latest generation can compete at the very top on hard maths. For a company that has spent heavily to catch up in AI, that is a meaningful public result.

But benchmarks have a well-known blind spot: they measure performance, not value for money. Carbon's score of 85.4% is better than Sol's 80.6%, a gap of under five percentage points, while its cost per problem is roughly 15 times higher. For most real uses, a slightly lower score at one-fifteenth of the price is the better deal. That is why the headline needs the cost figure next to it, and why a leaderboard win alone does not guarantee customers will follow.

The statistical caveats matter too. When confidence intervals overlap, the true gap between models may be much smaller than the headline numbers suggest, or even zero. And when part of a score is estimated rather than measured, and the model's most common failure is running far too long, the result looks less like a clean victory and more like a strong but expensive and sometimes slow model. None of that erases the achievement — it just puts it in its proper frame.

Analyst's Take

Our call is Mixed. Gemini 4 Carbon leading MathArena is a real achievement for Google and a reminder that the model race is not a one-company contest. But a first-place score that costs roughly 15 times as much per problem as the model it beat is a qualified win, and the overlapping confidence intervals mean the lead itself is not as firm as the headline suggests.

The cost gap is the heart of the matter. Benchmarks reward raw ability; customers reward ability per dollar. Until Google can bring Carbon's cost closer to Sol's, many users will keep choosing the cheaper model that scores nearly as well. The 11-hour run that blew past the time limit is a further warning that Carbon's approach can be slow and wasteful in ways a leaderboard does not capture.

Even so, this is a useful result for Google. Leading a respected benchmark, even at a high price, shows its models can compete at the top end, and cheaper versions of the same technology often follow. The next thing to watch is whether Google ships a lower-cost version that keeps most of the score. Reader poll: Bullish or bearish on GOOGL? Vote: Bullish, Bearish, Mixed, Too pricey to win.

Are you bullish or bearish on this story?

By The Numbers

85.4%Gemini 4 Carbon's expected performance on MathArena
82.8%Claude Fable 5.2's expected performance
80.6%GPT-6.1 Sol's expected performance, the previous leader
$15.70Cost of one run per problem on Gemini 4 Carbon
$1.07Cost of one run per problem on GPT-6.1 Sol
$40.20Cost of one run per problem on Claude Fable 5.2

FAQ

What is MathArena?

MathArena is a benchmark, or test, that scores AI models on hard maths problems and ranks them by expected performance.

Why does Gemini 4 Carbon's cost matter if it won?

A model that scores highest but costs roughly 15 times as much per problem may be too expensive for everyday use, so the win is less useful in practice than the headline suggests.

What are the doubts about Carbon's score?

Confidence intervals overlap with rivals, part of the score is statistically estimated rather than measured, and its most common failure was exceeding the time limit — in one case running a script for over 11 hours.

👁

Source: The Decoder (via MathArena)

Get the daily briefing. AI news with a markets lens — tickers, bullish/bearish reads, zero hype. Subscribe free on Substack

More on $GOOGL

All $GOOGL coverage →

More from AI Agent Update