Cheap, Open Models Are Eating the Usage Chart—But Traffic Is Not a Benchmark

A leaderboard can tell you what won. It cannot tell you what game was being played.

This week’s OpenRouter rankings look like a breakout for cheaper, open-weight models. Z.ai’s GLM 5.3 Flash climbed two places to No. 6 after processing 6.16 trillion tokens in the trailing seven-day window. Tencent’s Hy4 preview moved from No. 11 to No. 10 with 3.07 trillion tokens, while the free edition of MiniMax M3 rose to No. 12 on 2.57 trillion. At the same time, Arena AI’s preference-oriented Overall top 20 did not change at all.

That divergence is the story. The market is learning to separate models people prefer in blind comparisons from models developers are willing to route enormous workloads through. Those overlap, but they are not the same measurement—and treating them as interchangeable is how teams turn a useful adoption signal into a bad procurement decision.

Six trillion tokens buys a test, not a trophy

OpenRouter ranks individual model variants by prompt and completion tokens over a rolling seven-day period. It excludes private requests, and it lists free and paid variants separately. The chart therefore measures traffic through one routing platform, not unique users, request count, revenue, task success, or output quality. Even the token totals are not perfectly comparable: tokenizers split identical text differently, and verbose models can climb the chart by producing more text without completing more work.

The traffic is still meaningful. Trillions of tokens represent more than a benchmark launch or a well-amplified announcement. Developers are putting these endpoints into agents, coding tools, document pipelines, and experiments at a scale large enough to expose operational behavior. But the correct interpretation is “this model has earned evaluation capacity,” not “this model is now the sixth-best model available.”

GLM 5.3 Flash illustrates both the opportunity and the measurement problem. Z.ai describes it as a 320-billion-parameter mixture-of-experts model with 18 billion active parameters, native multimodality, a one-million-token context window, and MIT-licensed weights. That active-parameter count helps explain its hosted inference economics. It does not make the model equivalent to a dense 18B model for self-hosting: all 320B parameters still affect storage, loading, quantization strategy, and deployment complexity.

There is also an attribution wrinkle. Z.ai says GLM 5.3 Flash was previewed anonymously as Ox Alpha to collect real-world feedback. OpenRouter still lists Ox Alpha separately at No. 1 with 15.7 trillion weekly tokens, down from 20.7 trillion, while the named GLM endpoint appears at No. 6. Combining those rows might tell a useful model-family story, but silently doing so would invent a number the ranking itself does not publish. Anonymous previews are valuable product research; they also make adoption charts unusually easy to overread.

Hy4 arrives with strong economics and a promotional asterisk

Tencent’s Hy4 preview is larger still: 770B total parameters, 49B active, and a claimed context window beyond one million tokens. Tencent reports an average score of 2.99 out of 4.00 in an internal blind evaluation involving 163 experts and 203 engineering tasks, compared with 2.94 for Kimi K3 and 2.92 for GLM 5.3. Those are specific results, but they remain vendor-run results until the task set and evaluation procedure are reproducible outside Tencent.

The more concrete engineering claim may be in the serving stack. Tencent says work on Hy4 led to operator-fusion and communication changes that improved end-to-end throughput by 31.8% over its baseline. That matters because large-model competition is increasingly an inference-systems contest. Once several models are capable enough for routine agent steps, batching, caching, communication overhead, and reliable structured output can matter more to a production budget than a narrow benchmark lead.

Hy4’s API pricing makes the intended fight explicit: $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million cached tokens. Distribution spans Tencent products, Tencent Cloud TokenHub, and OpenRouter. WorkBuddy and CodeBuddy also include a two-week free launch window. Free access is a sensible way to accelerate evaluation, but it is also a confounder in any usage ranking. Launch traffic mixes genuine demand, curiosity, subsidized workloads, and developers running bake-offs they may never ship.

MiniMax M3 exposes another accounting issue. Its free variant processed 2.57 trillion tokens at No. 12, while the paid variant reached No. 16 with 1.52 trillion. Keeping endpoints separate is correct for routing and pricing decisions. It also understates family-level adoption when readers glance at only one row. Rankings look precise because they are ordered; the underlying product taxonomy is messier.

The production bake-off needs your traces, not someone else’s leaderboard

Engineering teams should use this churn as a discovery queue. Pull representative tasks from production: coding changes with tests, tool calls with strict schemas, retrieval over long documents, customer-support resolutions, or whatever actually pays the bill. Pin exact model IDs and versions. Disable silent fallback while evaluating, because a client that quietly substitutes another model can produce flattering results, confusing bills, and impossible-to-reproduce failures.

Then measure completed work rather than raw token throughput. Record task success, human correction rate, latency percentiles, input and output tokens, cache-hit rate, tool-call validity, retries, and cost per successful task. A cheap model that rambles, retries tools, or fails late in a workflow can cost more than an expensive model that finishes cleanly. A less prestigious model can also win decisively when it handles routine agent steps reliably at a fraction of the price.

For teams considering self-hosting, active parameters are only the beginning of the spreadsheet. Price the full weight footprint, GPU memory, quantization loss, interconnect requirements, KV-cache growth at realistic context lengths, scheduler behavior under concurrency, and the staff time required to operate the deployment. “One-million-token context” describes what an API accepts; it does not prove accurate retrieval across that window or that paying for it is rational. Test context degradation at the lengths you expect to use, not at the maximum printed on the model card.

Community discussion around both launches reflects that split. Developers are interested in low-cost agentic throughput and open weights, while local-inference practitioners point out that 320B and 770B models remain datacenter-class systems even with sparse activation. Reports of roughly 50 tokens per second at peak through OpenRouter are useful observations, but routing conditions, cache state, geography, and provider load can dominate a single measurement. Benchmark the endpoint from where your application runs.

The stable Arena chart supplies the needed counterweight. Arena asks humans which blind output they prefer; OpenRouter shows what traffic developers send through an aggregator. Preference does not equal production reliability, and traffic does not equal quality. Mature model selection needs both kinds of evidence plus internal evaluation data that neither public chart can provide.

Cheap inference is now powerful enough to move trillions of tokens before evaluation culture catches up. That is good news for builders: credible alternatives are arriving faster, and price competition is becoming operational rather than theoretical. The disciplined response is to route experiments quickly and promote models slowly. Tokens burned are evidence of attention. Work completed is evidence of value.

Sources: OpenRouter Rankings, Arena AI Leaderboard, Z.ai GLM-5.3-Flash announcement, Tencent Hy4 preview announcement, Hacker News discussion of GLM 5.3 Flash, Hacker News discussion of Hy4 preview