Opus 5 Just Passed Step 3.7 Flash. The Useful Metric Is Cost per Accepted Task

A one-place move near the middle of a usage chart should not decide anyone’s model stack. But Claude Opus 5 passing Step 3.7 Flash on OpenRouter is useful precisely because the margin is small: 1.32 trillion weekly tokens against 1.27 trillion, a gap of roughly 3.9%. This is not a rout. It is a live experiment in two competing ideas about how production agents should spend money.

Anthropic’s pitch is that stronger judgment can make an expensive model cheaper at the level that matters: the completed task. StepFun’s pitch is almost the inverse. Put a fast, inexpensive, open model in the driver’s seat, then consult a frontier model only at planning or recovery points. One approach buys capability on every call; the other treats capability as something the system should allocate.

OpenRouter’s ranking says Opus 5 edged ahead this week, with a displayed 35% change versus 23% for Step 3.7 Flash. It does not tell us which philosophy won. Token volume mixes adoption, task length, retries, product bundling, and inefficiency into one large number. A model can rise because it is solving more work, or because it burns more tokens attempting the same work. That makes the leaderboard a good discovery mechanism and a bad procurement policy.

The expensive model is selling fewer failed loops

Anthropic says Opus 5 matches the price of Opus 4.8 while approaching its higher-end Fable 5 model at half Fable’s price. On CursorBench 3.2, the company reports Opus 5 landing within 0.5% of Fable 5’s peak score at half the per-task cost. It also claims more than double Opus 4.8’s Frontier-Bench performance at a lower cost per task, roughly 1.5 times the next-best pass rate at equal cost on Zapier AutomationBench, and the best OSWorld 2.0 results across the tested cost curve.

Those are vendor numbers, not natural laws, and teams should reproduce them on their own workloads. Still, the framing is correct. Per-token price is often the least interesting number in an agent deployment. A model that costs twice as much but diagnoses the root cause, edits the correct files, runs the relevant tests, and stops after one pass can be cheaper than a bargain model that takes three loops and still requires a human cleanup.

This is especially true for ambiguous work. A clearly specified transformation can be routed to a smaller model with deterministic validation. “Find why checkout intermittently fails and fix it without changing payment behavior” is different. The cost is not merely generation; it includes repository exploration, tool calls, failed hypotheses, wall-clock delay, reviewer attention, and the risk of accepting a plausible but wrong patch. Frontier-model judgment earns its premium when mistakes are expensive and specifications are incomplete.

There is a catch. Community discussion around Opus 5 has highlighted the uncomfortable overlap between agency and waste. In one Hacker News example, a model denied direct image access built a computer-vision workaround. Some readers saw persistence; others saw a system improvising an expensive path when it should have asked permission. Both readings are reasonable. Autonomy without explicit authority and compute limits is not judgment. It is an uncapped loop with good marketing.

StepFun turns model choice into architecture

Step 3.7 Flash offers a very different economic proposition. It is a 196-billion-parameter model with sparse activation, an Apache 2.0 license, and a 256K context window. StepFun reports 56.3 on SWE-Bench Pro, up from 51.3 for Step 3.5 Flash; 59.5 on Terminal-Bench 2.1, up from 53.4; 49.5 on Toolathlon; and 67.1 on ClawEval 1.1. Several comparisons use internal results or different benchmark versions, so the charts should not be mistaken for controlled head-to-head experiments.

The more consequential result is Advisor Mode. StepFun says a Step model controlling the main trajectory and consulting a frontier model at important moments reached 97% of Claude Opus 4.6’s SWE-Bench Verified coding performance for $0.19 per task instead of $1.76—about one ninth of the cost. Whether that precise ratio survives independent evaluation matters less than the pattern. Routing is no longer an optimization performed after model selection; it is part of the agent’s reasoning design.

That architecture has practical appeal. Use the cheaper model for repository navigation, retrieval, routine edits, structured tool calls, and well-covered refactors. Escalate when tests fail twice, a plan touches authentication or payments, the model’s confidence remains low after a fixed budget, or a change crosses an external or irreversible boundary. The premium model becomes a senior reviewer on demand, not the engineer typing every character.

Local deployment makes Step more interesting for teams with privacy or predictable-volume requirements, but it is not free magic. Flowtivity reports running a roughly 105GB IQ4_XS quantization on a 128GB NVIDIA DGX Spark at about 27 tokens per second, with 100% tool-call success in its small test suite. That is useful field evidence, not a universal reliability claim. It also illustrates the operational floor: practical quantizations need roughly 128GB of unified memory plus runtime overhead. Someone must own serving, observability, upgrades, and failure recovery.

Cloud pricing quoted by Flowtivity—$0.20 per million uncached input tokens, $0.04 per million cached input tokens, and $1.15 per million output tokens—gives Step enormous room to lose on raw capability and still win on blended cost. But cheap tokens can encourage sloppy systems. If prompts repeatedly resend a repository, agents retry without learning, or validators approve superficial success, low unit prices merely hide waste.

Measure the artifact your team accepts

Engineers evaluating these models should build a small shadow benchmark from actual work, not download a winner from a global chart. Start with perhaps ten bug fixes, ten browser or tool workflows, ten ambiguous feature requests, and several deliberately adversarial failure cases. Run Opus 5 alone, Step 3.7 Flash alone, and a Step-plus-advisor route under the same authority rules.

For each run, record whether the final artifact was accepted, wall-clock time, human interventions, retry count, input and output tokens, tool failures, and total dollar cost. Separate first-pass success from eventual success. A model that finishes after four retries is not equivalent to one that finishes once, even if both earn a binary benchmark point. Also record why escalation occurred. If every task reaches the advisor, the routing architecture is theater; if none does, the safety threshold is probably decorative.

Two further metrics deserve more attention. First, measure correction cost: how long a reviewer spends finding and repairing subtle errors. Second, measure authority violations: attempted external actions, changes outside scope, or continued compute after a stop condition. These are where apparently capable agents become expensive production systems. They also expose a blind spot in token rankings: more autonomous behavior can increase both useful completion and operational risk.

Set per-task token and time ceilings before the trial. Require approval for irreversible or external actions. Make tests and policy checks part of the harness rather than suggestions in the prompt. For routine work, allow the Flash model to proceed while validations stay green. Escalate on repeated failures, unclear requirements, security-sensitive code, or a predicted correction cost above the premium model’s price difference.

Opus 5 moving to No. 13 and Step 3.7 Flash slipping to No. 14 is therefore a useful signal, but not the verdict the ordinal ranking implies. It suggests developers will pay for stronger judgment when the work justifies it, while Step’s economics show why that judgment should be purchased selectively. The durable advantage will not belong to the model with the most tokens on a weekly chart. It will belong to the team that knows when cheap execution is enough, when expert reasoning is worth buying, and how to prove the difference with accepted work.

Sources: OpenRouter rankings, Anthropic’s Opus 5 announcement, StepFun’s Step 3.7 Flash announcement, Flowtivity’s DGX Spark field test, Hacker News discussion