Running Qwen Behind Claude Code Is the Local-Agent Wedge Alibaba Should Care About

The interesting part of the latest Qwen/Claude Code field guide is not that someone made a local model answer coding prompts. That stopped being novel somewhere around the third weekend everybody spent compiling llama.cpp. The useful part is more specific: a refreshed GitHub Gist shows what it actually takes to put Qwen behind a Claude-shaped coding workflow without pretending the model is the whole product.

The setup is direct enough to describe in one line: Claude Code → llama-server on localhost:8131 → Qwen3.5-35B-A3B GGUF on Apple Silicon. Claude Code is configured through Anthropic-style environment variables — ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL, the default Opus/Sonnet/Haiku aliases, and CLAUDE_CODE_SUBAGENT_MODEL — while llama.cpp serves the quantized Qwen model over HTTP. In other words, the agent UI still thinks it is talking to a cloud API. The traffic goes to a local Mac instead.

That is the wedge Alibaba should care about. Most teams do not wake up wanting another model endpoint. They want their existing agent workflows to become cheaper, more private, more controllable, or all three. If Qwen can sit behind the tools developers already use — Claude Code, OpenCode, Codex-style terminals, OpenClaw sessions, local routers — then Qwen is not merely competing at the model layer. It is competing for the runtime slot where daily engineering work happens.

“Anthropic-compatible” is a runtime contract, not a sticker

The Gist is valuable because it exposes all the boring seams that marketing usually rounds off. The author recommends serving unsloth/Qwen3.5-35B-A3B-GGUF:Q4_K_M with Metal acceleration, -ngl 999 to offload layers, a 131,072-token context on larger machines, --parallel 1, Flash Attention, KV-cache quantization with q8_0, --keep 1024, --mlock, and --no-mmap. On 16 GB machines, the guide suggests dropping context to 32,768 tokens. The first model download is roughly 20 GB; the stated practical floor is Apple Silicon with 24 GB unified memory, while 64 GB is the “this stops feeling like a stunt” tier.

Those numbers matter because local coding agents fail less often from grand theoretical limitations than from operational friction. Context too large? The model server stalls or swaps. Wrong chat template? Tool-use output gets weird. Prompt caching not behaving? Follow-up latency turns a flow state into a coffee break. The guide’s most consequential flag may be --swa-full, which it calls out for Qwen3.5’s sliding-window attention behavior; without it, follow-up requests can reprocess too much context instead of benefiting from cache behavior. That is not leaderboard material. It is the difference between “usable for a repo session” and “nice demo, never again.”

The same goes for disabling Qwen’s thinking output with --chat-template-kwargs '{"enable_thinking": false}'. Unsloth’s model card also notes that Qwen3.5 models operate in thinking mode by default and documents disabling that behavior. In a coding-agent stack, this is not just a taste preference. Claude Code already has its own planning loop, tool loop, and response expectations. Hidden or explicit reasoning tokens from the model can waste local inference budget, pollute responses, or confuse downstream parsing. When a model is sitting behind another agent runtime, “more reasoning text” is not automatically better. Sometimes it is impedance mismatch wearing a lab coat.

The model is impressive; the adapter is the product

Qwen3.5-35B-A3B is a serious open-weight target for this experiment. Unsloth’s model card lists 35 billion total parameters with 3 billion activated, 256 experts with 8 routed plus 1 shared active, native 262,144-token context extendable to 1,010,000 tokens, and compatibility across runtimes including Transformers, vLLM, SGLang, KTransformers, and GGUF. The same card says refreshed GGUFs improved chat, coding, long-context, and tool-calling use cases, and specifically mentions improvements via Claude Code and Codex.

But the article practitioners actually need is not “Qwen can code.” It is “Qwen can survive the adapter.” Claude Code-shaped tooling assumes a certain API grammar: model names, auth checks, streaming behavior, message formatting, tool-call-shaped outputs, cancellation, and sometimes prompt-caching expectations. A localhost server can accept requests and still fail the workflow in death-by-paper-cut ways. An Anthropic-compatible base URL is the start of the contract, not the contract itself.

This is why the local-agent story is becoming a router story. The Gist includes a cclocal helper for switching ports and points at possible alternate backends such as Qwen3-Coder, GLM-4.7-Flash, GPT-OSS-20B, and Qwen3-Coder-Next. Once developers start doing this seriously, “which model?” becomes only one dimension. They will also want per-project model routing, fallback when the local server is cold, a way to compare hosted DashScope or OpenRouter behavior against local GGUF behavior, and clean separation between fast autocomplete-like edits and heavy repo reasoning. The winning Qwen workflow may not be one model running forever. It may be a small local routing layer that knows when Qwen3.5 is enough, when a coder-specialized model is better, and when the task should go to a hosted endpoint with a real SLA.

Local does not mean governed

The Gist’s permissions example is also a useful teaching moment because it is both practical and risky. It pre-approves a long list of shell and file tools so Claude Code does not ask for confirmation on every mundane operation, while denying reads of .env, .env.*, and secrets/**. The instinct is right: agent workflows need policy, and secret files should not be casual context snacks.

The problem is that broad allowlists are where “local” gets mistaken for “safe.” The example includes permissive shell patterns such as Bash(rm *) and Bash(env *). That should make operators pause. Qwen Code itself recently hardened shell policy around env because it can act as a command proxy rather than a harmless read-only inspection tool. A local model removes cloud egress and usage-based billing. It does not remove prompt injection, accidental destructive commands, confused-deputy behavior, or the need for approvals around writes, deletes, package scripts, and network calls.

If anything, local agents demand clearer policy because there is no vendor safety layer to blame. The model is on your machine, the repo is on your disk, and the permissions file is your blast-radius document. Treat it that way. Start with read-only tools. Add edit permissions only after the model behaves in a throwaway repo. Keep secrets denied. Avoid blanket shell approvals. If you do allow common commands, separate inspection commands from mutation commands. “No API bills” is a nice promise. “No incident report” is better.

What engineers should actually test

The right evaluation is not a vibe check where the model writes a toy function. Put the stack through boring failure modes. Measure cold-start time after loading the 20 GB model. Measure first-token latency and follow-up latency after cache warmup. Run a medium repo task that requires reading multiple files, making a small change, running tests, and revising. Watch tool-call reliability, not just final prose. Cancel a task mid-run. Ask it to inspect a file it should not read. Try a long context session until memory pressure appears. Then compare the same workflow against hosted Qwen Code via Alibaba Cloud Coding Plan, a standard OpenAI-compatible DashScope endpoint, and whatever cloud agent you currently pay for.

Also test the human workflow. Can another developer reproduce the setup without inheriting your shell state? Are the llama.cpp flags captured in a script or hidden in terminal history? Does the team know which port maps to which model? Is auto-update disabled intentionally, or did it drift? Is telemetry disabled because privacy matters, or because somebody copied a snippet without understanding the trade-off? Local AI stacks age like any other infrastructure: dependencies update, model cards change, quantizations improve, CLIs alter their auth paths, and yesterday’s “works on my M3 Max” becomes tomorrow’s onboarding tax.

That is why this Qwen guide is more interesting than another model announcement. It shows Qwen becoming useful at the point where developers already spend their time: inside agentic coding loops with files, tools, permissions, and latency constraints. Alibaba’s opportunity is not merely to publish stronger weights. It is to make Qwen boringly compatible with the workflows developers refuse to give up.

The take: local Qwen behind Claude Code is not a Claude killer. It is a pressure test for the next phase of coding agents. The model has to be good, but the adapter, cache behavior, permission policy, hardware envelope, and routing story decide whether it earns a permanent spot in an engineer’s day. Localhost is attractive because it removes invoices and data egress. It gets dangerous when teams confuse “runs on my Mac” with “ready for my repo.”

Sources: GitHub Gist: How to Run Qwen3.5 Locally With Claude Code, Unsloth Qwen3.5-35B-A3B GGUF model card, llama.cpp, Qwen Code authentication docs