The Model People Prefer Is Not the Model Builders Actually Run

The most important model leaderboard result this week is not that Xiaomi moved into first place. It is that the model most heavily used through OpenRouter is nowhere near the models people most often prefer in Arena AI’s blind comparisons. That gap is no statistical footnote. It is the clearest evidence yet that the market for attractive answers and the market for economically viable software are separating.

Xiaomi’s MiMo-V2.5 processed 9.6 trillion tokens on OpenRouter’s weekly chart, up 19% in the displayed comparison, displacing Tencent’s free Hy3 endpoint at 8.54 trillion tokens. DeepSeek V4 Flash, in third place, was well behind at 5.48 trillion. Meanwhile, Arena’s text top 20 did not move: Claude Fable 5 remained first at 1507 Elo, and Anthropic held six of the first seven positions.

Both rankings can be accurate because they answer different questions. Arena asks a human which of two anonymous responses they prefer. OpenRouter counts tokens sent through one routing marketplace, where price, promotions, integration defaults, context length and the verbosity of agent loops all affect the result. One measures taste under a controlled interaction; the other captures a slice of operational demand. Calling either one “the best model leaderboard” discards the most useful part of the data.

A free-token race can still reveal paid demand

MiMo’s lead needs an asterisk, but not a dismissal. Xiaomi released MiMo-V2.5 and MiMo-V2.5-Pro weights under the MIT license and says it offered 100 trillion free tokens through its Orbit program. It also integrated the models with OpenCode, Hermes Agent and KiloCode. A subsidy of that size is not background noise: it lowers the cost of experimentation and can push a model into workflows before procurement, evaluation and habit have caught up.

That means 9.6 trillion tokens should not be translated into “users selected MiMo at the market-clearing price.” It does mean developers found enough utility to consume those tokens. Free capacity does not automatically create working integrations, successful tool calls or repeat use. The better question is how much traffic remains once promotional economics stop doing the recruiting.

Tencent provides a useful second signal. The free Hy3 endpoint remained No. 2 with 8.54 trillion weekly tokens, while the paid endpoint climbed five places to No. 10, reaching 1.35 trillion tokens after OpenRouter displayed growth above 999%. Combined, the two endpoints account for roughly 9.89 trillion tokens. That is not a legitimate single-model rank—the endpoints have different prices and possibly different workload mixes—but it shows how a chart organized by endpoint can conceal vendor-level adoption.

Paid Hy3 is therefore more interesting than the headline winner. Launch-week growth above 999% is inherently unstable, but 1.35 trillion paid tokens suggest that at least some users are willing to keep testing the model when the bill is visible. Community discussion has centered on reported OpenRouter pricing of $0.14 per million input tokens and $0.58 per million output tokens, plus a free tier. For an agent that plans, calls tools, retries and produces long intermediate traces, output price can matter more than a small benchmark advantage.

Sparse activation does not make 295 billion parameters disappear

Hy3’s architecture explains part of the interest and part of the deployment confusion. It is a 295-billion-parameter mixture-of-experts model with 21 billion active parameters, plus 3.8 billion MTP parameters. Tencent specifies 192 experts with eight active across 80 layers and a 256,000-token context window. The model is available under Apache 2.0.

Those 21 billion active parameters reduce compute per generated token; they do not turn Hy3 into a conventional 21B model for memory planning. Tencent recommends eight large-memory H20-class GPUs for serving the full weights with tensor parallelism through vLLM or SGLang. Sparse models can offer excellent inference economics at scale, but inactive experts still have to live somewhere. “Open weights” removes licensing and provider lock-in; it does not remove capacity planning, quantization work, batching decisions or pager duty.

This distinction matters because local-model enthusiasm is strong. An indexed LocalLLaMA thread titled “Tencent-HY3 is the real deal on 128GB” attracted 286 votes and 155 comments. That is meaningful practitioner curiosity, not an independent benchmark. Quantized deployments may be entirely sensible for experimentation or privacy-sensitive work, but engineering teams should compare their actual throughput and quality against hosted endpoints before buying hardware to satisfy an architectural aesthetic.

Tencent’s quality claims are promising and appropriately testable. In a blind evaluation by 270 experts, the company reports Hy3 scoring 2.67 out of 4 against GLM-5.1’s 2.51, with its largest advantages in frontend development, data and storage work, and CI/CD. It also reports hallucination rates dropping from 12.5% to 5.4%, commonsense errors from 25.4% to 12.7%, and multi-turn issues from 17.4% to 7.9%. These are vendor-run results, not independent proof, but they name failure modes that production teams can reproduce instead of hiding behind an average score.

One claim deserves particular attention: Hy3’s reported SWE-bench Verified accuracy varies by less than four percentage points across CodeBuddy, Cline and KiloCode scaffolds. Agent performance is often as much a property of the harness as the underlying model. A model that remains stable across scaffolds may be more valuable than one that wins under a single carefully tuned prompt and collapses when tool schemas or retry policies change.

Stop buying models by rank

Six of OpenRouter’s top 11 entries come from Xiaomi, Tencent, DeepSeek, Z.ai, MiniMax and StepFun. On Arena, Anthropic dominates the preference frontier. The practical reading is not that open Chinese models have “beaten” Claude, or that Arena users are out of touch. It is that quality, price and workload shape different leaderboards, and agent workloads amplify small differences in token economics.

A chat response usually ends after one exchange. A coding or operations agent may generate a plan, inspect files, invoke several tools, recover from malformed calls, delegate subtasks and rewrite its answer. A model that costs slightly less per token can be dramatically cheaper per completed request—but only if it does not erase the savings with retries and human correction. Conversely, a premium model can be the economical choice when it succeeds on the first attempt.

Teams considering MiMo or Hy3 should run a shadow evaluation on 100 to 300 real tasks before changing a production router. Preserve tool schemas, system prompts and scaffold versions. Score completed outcomes, not how polished the prose sounds. At minimum, record tool-call validity, retry count, wall-clock latency, input and output tokens per successful task, hallucinated citations, human correction time and the percentage of jobs handed to a fallback model.

Test reasoning modes separately. Hy3 exposes no_think, low and high; routing every routine request through high reasoning can turn bargain tokens into an expensive trace generator. Test long-context behavior adversarially as well. A 256K or one-million-token context window is a capacity ceiling, not evidence that the model will preserve the right constraint near that ceiling. Place conflicting instructions and required details at different positions, then measure retrieval and compliance.

Most teams should begin with two hosted providers behind an OpenAI-compatible abstraction. Pin model identifiers, log the provider and version for every run, and maintain a tested fallback. Self-host when sustained utilization, data residency, customization or predictable latency justifies owning the operational stack—not because an Apache or MIT license makes the GPUs free.

The next few weeks will distinguish adoption from launch traffic. Watch whether paid Hy3 stays in OpenRouter’s top 15 after the initial spike and whether MiMo holds first place once Xiaomi’s incentives fade into normal ecosystem support. Also watch outcome-based evidence: independent scaffold tests, failure recovery rates and cost per completed task. Token volume alone rewards verbose loops; preference Elo alone rewards answers humans like in a narrow comparison.

Xiaomi won this week’s usage chart. Anthropic still owns the preference chart. The mature conclusion is not to crown two winners, but to stop asking one leaderboard to choose infrastructure for you. Use preference data to identify capable candidates, deployment data to find economically credible ones, and your own evaluation to decide which model deserves production traffic. Worship neither table; instrument the work.

Sources: OpenRouter Rankings, Xiaomi MiMo-V2.5 open-source announcement, Tencent Hy3 announcement, Hy3 model card and deployment guide, Arena AI leaderboard