The Models Developers Pay to Run Are Not the Models They Prefer

The most important model ranking this week is not a ranking of intelligence. It is a bill.

OpenRouter’s latest usage table shows developers sending 9.95 trillion weekly tokens through Xiaomi’s MiMo-V2.5, 5.81 trillion through DeepSeek V4 Flash, and a combined 7.70 trillion through Tencent’s free and paid Hy3 routes. Meanwhile, Arena AI’s preference leaderboard did not move at all: Claude Fable 5 remains first, followed by two Claude Opus thinking variants. Put the two tables side by side and the fashionable question—“Which model is best?”—starts to look badly specified.

Arena measures which response people prefer in head-to-head comparisons. OpenRouter measures token consumption. Neither tells an engineering team whether a model completes its work reliably, but the divergence is useful. It says the models people admire and the models software actually keeps busy are increasingly different populations.

The second-place change is really a price-performance vote

DeepSeek V4 Flash passed the free Hy3 endpoint to take second place on OpenRouter, rising from 5.59 trillion to 5.81 trillion weekly tokens. Free Hy3 moved the other way, falling from 6.79 trillion to 4.93 trillion. That looks like a simple rank swap until the paid Hy3 route enters the picture: it climbed two places, from eighth to sixth, and increased from 2.06 trillion to 2.77 trillion tokens in a day’s snapshot. OpenRouter displays its week-over-week growth as greater than 999%.

The percentage deserves skepticism because enormous growth often starts with a tiny denominator. The absolute volume is harder to dismiss. A free endpoint can attract benchmarking, weekend experiments, and applications designed around somebody else paying the inference bill. A paid route accumulating trillions of tokens suggests that at least some users have crossed the line from curiosity to metered use.

That does not prove Hy3 is winning production. Launch credits, routing defaults, provider integrations, and a few long-context workloads can distort a token chart. Tencent also has distribution outside OpenRouter: it says Hy3 has been integrated into CodeBuddy and WorkBuddy, Yuanbao, Marvis, ima, WeChat scenarios, and games. The company reports a 20-fold increase in average daily token consumption after the preview launched and a sixfold increase in WorkBuddy users actively selecting it. Those are vendor numbers, not audited adoption data, but they make the paid-route jump more plausible than a leaderboard glitch.

The better test comes over the next several weeks. If paid Hy3 stays in the top ten after launch incentives normalize, while its latency and error rates remain usable under load, the story is durable. If it collapses when the free route or promotional distribution changes, today’s spike was acquisition rather than retention.

Sparse activation is becoming the commercial interface

The top of OpenRouter is now a showcase for sparse Mixture-of-Experts economics. MiMo-V2.5 has 310 billion total parameters but activates 15 billion for each token. DeepSeek V4 Flash has 284 billion total and 13 billion active. Hy3 has 295 billion total and activates 21 billion. Their advertised context windows range from Hy3’s 256K to one million tokens for MiMo and DeepSeek.

Those active-parameter counts are not a magic conversion into the cost profile of a dense 13B or 21B model. Expert routing, memory movement, attention over long contexts, and distributed serving all consume real hardware. But sparse activation changes what vendors can sell: broad capability from a large parameter pool without paying the full compute cost of activating the entire model on every token. That creates enough price and throughput headroom for developers to route high-volume classification, search, editing, and agent loops away from premium frontier APIs.

Xiaomi says MiMo-V2.5 was trained on 48 trillion tokens, supports native vision and audio, and reaches a one-million-token context. It reports a 62.3 score on the general subset of its Claw-Eval agent benchmark and says the model matches MiMo-V2.5-Pro on an internal coding benchmark at half the cost. DeepSeek similarly says V4 Flash approaches V4 Pro in reasoning and matches it on simple-agent performance. These claims are useful hypotheses for an evaluation queue. They are not reasons to merge a routing change.

Community reports explain why. In practitioner discussion, one evaluator found DeepSeek V4 Pro stronger for one-shot coding but unreliable inside a custom agent harness; another reported that Flash handled a similar harness cleanly. MiMo users have described the non-Pro model as cheap and sufficient for roughly 90% of their tasks, while another needed to add a concrete Bash tool-call example to prevent malformed calls. A user-reported MiMo result of 55% on Terminal Bench 2.0 sounds impressive until you remember that tool schemas, prompts, retries, and environment setup are part of the tested system. The harness is not incidental plumbing. For agents, it is part of the model.

Tokens are activity, not completed work

OpenRouter’s leaderboard rewards consumption. A model that reads a 500,000-token repository context twice and fails can generate more chart activity than a model that reads a targeted 40,000-token retrieval bundle and completes the change. Long agent trajectories, verbose reasoning, retries, and cache misses all inflate token volume. Popularity can therefore rise while unit economics deteriorate.

That is the first practical lesson for engineering teams: measure completed tasks per dollar and per minute, not price per token in isolation. Your evaluation set should replay actual system prompts, tool definitions, repository shapes, structured-output validators, retry policies, and failure recovery. Track first-pass completion, repair count, wall-clock latency, invalid tool calls, and human intervention. A cheap model that needs three repairs may be more expensive than a premium model that succeeds once.

The second lesson is to resist turning a one-million-token context window into an architecture. Instrument input growth across agent turns, prompt-cache hit rates, tool-result truncation, and quality by context bucket. Retrieval and summarization still matter. Sending an entire codebase because the API accepts it is the inference equivalent of using a data warehouse as a queue: technically possible, economically revealing.

The third lesson is to route explicitly. Efficient models such as DeepSeek V4 Flash, MiMo-V2.5, or Hy3 are credible candidates for classification, mechanical edits, repository search, and first-pass tool work. Escalate ambiguous refactors, repeated tool failures, security-sensitive changes, and difficult reasoning to a stronger fallback. Pin versions, validate structured output, cap repair loops, and shadow-test replacements before changing the default. Vendor benchmarks belong in model discovery; private acceptance tests belong in the release gate.

The narrow one-billion-token gap at the bottom of OpenRouter’s top 20 makes the same point in miniature: GPT-5.6 Sol moved to nineteenth with 572 billion weekly tokens, edging MiMo-V2.5-Pro at 571 billion. Rank can change on a rounding error while the engineering reality remains identical. Step 3.7 Flash and Kimi K3 also rose, displacing Claude Opus 4.8 and Claude Sonnet 5 by one position each. This is a fluid market for workload allocation, not a stable hierarchy of minds.

The open-model race has moved beyond parameter-count theater. The competitive question is who can turn sparse activation, long context, and sufficiently reliable tool use into the lowest cost per completed job. OpenRouter’s trillions of tokens show developers are testing that proposition at scale. Arena’s static preference table supplies the necessary warning: usage is not quality, just as quality without viable economics is not a production plan.

The models at the top of the bill have earned a trial, not a promotion. Ship the router, keep the fallback, and make every candidate pass the harness it will actually have to survive.

Sources: OpenRouter AI Model Rankings, Arena AI Text Leaderboard, Xiaomi MiMo-V2.5 release, Tencent Hy3 release, DeepSeek V4 release notes, Hacker News practitioner discussion