The LLM Leaderboard Is Measuring Distribution, Not Intelligence
The most important model release this week has a date stamp for a name. DeepSeek V4.1 Flash 1017 moved from sixth to third place on OpenRouter in one daily snapshot, adding roughly 4.16 trillion tokens to its trailing-week total. It now sits at 11.5 trillion tokens, just 0.1 trillion behind GLM 5.3 Flash in second place. That is an extraordinary amount of production-shaped attention for what DeepSeek calls a “minor update.”
But the lesson is not that DeepSeek has suddenly produced the third-best model. OpenRouter ranks tokens processed, not answer quality, completed tasks, unique users, revenue, or customer satisfaction. A long-context model can climb by consuming more tokens; a free endpoint can climb by removing the purchasing decision; a verbose model can look popular while wasting everyone’s time. The ranking is demand telemetry. Read correctly, it tells us where developers are sending workloads—not whether those workloads came back solved.
This week’s table is unusually useful because it shows two different distribution strategies rewriting the top ten at once. DeepSeek’s dated checkpoint rose three places to number three. MiniMax M3.1 Flash (free) entered at number ten with 4.97 trillion weekly tokens, though MiniMax says the model previously ran anonymously as Space Bunny and processed more than 200 trillion tokens before the reveal. Meanwhile Step 5 Preview fell from fourth to eighth as its rolling-week volume dropped from 8.22 trillion to 5.82 trillion, a 61% week-over-week decline. Rankings that look settled on Monday can become archaeology by Thursday.
A “minor update” with major migration economics
DeepSeek’s benchmark claims help explain the interest, particularly for coding agents. The company reports DeepSWE v1.1 rising from 74.2 to 77.0, Terminal-Bench 4.0 from 31.2 to 42.3, ProgramBench from 20.3 to 29.1, and Automation-Bench from 54.8 to 60.5. On Agents’ Last Exam, the checkpoint moves from 31.8 to 35.4. Those are vendor results, and the named harnesses matter: DeepSeek used Claude Code for Terminal-Bench and Mini-SWE-V2 for DeepSWE. The scores evaluate a model-harness pair, not a disembodied intelligence that will perform identically inside every company’s agent stack.
The more convincing adoption argument is operational. DeepSeek kept the existing deepseek-flash API route and pricing, offers a one-million-token context window, and released the weights under the MIT license. Existing callers get a low-friction upgrade, while teams with stricter security or reproducibility requirements can download and pin the checkpoint. DeepSeek also claims the new version produces shorter outputs and finishes tasks faster. If that holds outside its own evaluation setup, it improves the metric procurement teams should actually care about: cost per accepted task.
That convenient alias is also a production hazard. If deepseek-flash silently advances to a new checkpoint, application behavior can change without a deployment, pull request, or configuration diff. A “minor” model revision can alter tool selection, output structure, retry frequency, and latency tails. Teams qualifying 1017 should use the dated model identifier where their provider exposes it, save representative traces, and promote the moving alias only after the new checkpoint clears the same regression gates as a code release.
Self-hosters have another sharp edge to inspect. The model card recommends temperature 1.0, top-p 1.0, a one-million-token context window, and a maximum output allowance of at least 256,000 tokens. Yet the prompt format is not supplied as a conventional Jinja chat template. DeepSeek instead provides a Python reference encoder and deepseek-recipe, including Rust libraries and Python bindings for messages, tool calls, images, reasoning controls, and streaming parsing. That is capable infrastructure, but it means a naïve “load weights, paste template, benchmark” setup can fail at the protocol boundary before the model gets a fair test.
MiniMax won distribution before it finished the launch
MiniMax’s number-ten debut is impressive and easy to misstate. The free M3.1 Flash endpoint recorded 4.97 trillion trailing-week tokens, while OpenRouter’s daily view put it second with 3.91 trillion tokens and 272% day-over-day growth. But this was not a cold start. MiniMax says M3.1 Flash was the anonymous Space Bunny preview, which had already processed more than 200 trillion tokens. “New” describes the public identity on the leaderboard, not the beginning of demand.
The price is designed to make experimentation painless: $0.15 per million input tokens, $0.60 per million output tokens, and $0.015 per million cached input tokens, with a one-million-token context window. MiniMax claims output above 200 tokens per second in everyday use, median P50 throughput above 170 tokens per second, nearly three-times-faster inference, and 70% lower agent-task cost than its predecessor. It also reports 73.7% on DeepSWE v1.1, 76.2 on BankerToolBench, 61.7 on OSWorld V2, 29.6 on Agents’ Last Exam, and 82% on CyberGym.
Those claims deserve a pilot, not a procurement decision. On the two published rows that line up cleanly with DeepSeek 1017, MiniMax trails: 73.7 versus 77.0 on DeepSWE v1.1 and 29.6 versus 35.4 on Agents’ Last Exam. Other results may reveal genuine strengths, but benchmark names alone do not establish comparability when versions, subsets, judges, and tool harnesses differ. BankerToolBench is reportedly judged by an older MiniMax model, introducing another dependency into the number. An 82% vendor CyberGym score is a testing lead; it is not permission to grant an agent unsupervised access to production.
The missing weights are equally important. MiniMax says they “will be released later,” without a date, license, or downloadable checkpoint. For low-risk API workloads, that may be acceptable. For regulated data, reproducible builds, offline recovery, redistribution, or local fine-tuning, a future promise is not an artifact procurement can approve. DeepSeek’s MIT checkpoint is materially more concrete today.
Measure the task, not the token
The leaderboard’s volatility suggests a better evaluation plan for engineering teams. Build a fixed canary set from real work: repository-scale fixes, tool calls, document extraction, support classification, or whichever tasks actually consume budget. For every run, capture pass rate, input and output tokens, cache hits, retries, malformed tool calls, time to first token, wall-clock completion time, and human correction minutes. Then calculate cost per accepted task. A model charging half as much per token but looping twice, failing tool calls, or occupying an engineer for longer is not cheaper.
Route models by workload class rather than selecting one corporate champion. MiniMax’s pricing and cached-input rate may suit long, repetitive contexts. DeepSeek 1017’s benchmark pattern makes it a plausible candidate for coding and tool-heavy work, particularly where open weights or checkpoint pinning matter. Premium models may still win on tasks where a failed attempt is expensive. The router should encode those boundaries explicitly, with retry caps and fallbacks, instead of treating a public top-ten list as architecture.
There is also a useful interpretation of the base DeepSeek V4.1 Flash remaining first at 27.3 trillion tokens while falling 32% week over week: variant-level rankings can split what is economically one migration. Some demand is likely moving from the base name to 1017, and OpenRouter counts them separately. Add the two together and DeepSeek’s footprint looks dominant; view them independently and the new checkpoint looks like a sudden challenger. Neither reading is wrong, but only the combined view explains the release dynamics.
Arena’s preference leaderboard could not be independently refreshed for this edition because its security verification blocked both collection paths, so no stale rows are being substituted. That absence is worth stating because usage and preference answer different questions. OpenRouter tells us what developers are trying at scale. It cannot tell us which answer human evaluators prefer, and neither signal can replace traces from your own application.
The editorial verdict: this is not a horse race between model logos. It is evidence that compatibility, free distribution, open licensing, and serving economics can move trillions of tokens faster than benchmark discourse can stabilize. DeepSeek turned a dated patch into a near-number-two workload in days; MiniMax turned an anonymous preview into a top-ten launch. Builders should take the signal seriously, pin what they test, and refuse to confuse cheap tokens with completed work.
Sources: OpenRouter weekly rankings, DeepSeek announcement, DeepSeek model card, MiniMax launch post, cross-vendor benchmark audit