llama.cpp Adds Qwen-VL Video Plumbing and Fixes Reasoning Leakage Where Local Models Actually Break

The least glamorous local-AI work is usually the work that decides whether local AI is usable. A model card can promise a better Qwen, a smaller quant, a longer context window, or a more capable reasoning path. Then the serving stack has to make that promise survive contact with chat templates, video frames, reasoning channels, tool calls, GPU offload, and every awkward assumption the cloud API used to hide.

That is why the June 6 llama.cpp release stream is worth paying attention to. It is not a launch-day spectacle. It is plumbing. But two merged changes capture where local model deployment is going: Qwen-VL-style frame-merge support in multimodal tooling, and a fix for LFM2/LFM2.5 reasoning content leaking into visible chat content when reasoning is disabled. One is about making multimodal local inference faster. The other is about keeping model internals out of the user-visible channel. Both are production problems, not hobbyist trivia.

The releases were verified through GitHub’s release and pull-request stream, with builds from b9537 through b9544 published on June 6, 2026; b9544 landed at 21:20 UTC. The interesting commits are 31e8249, merged at 19:17 UTC, for Qwen-VL frame merge support, and 98d5e8b, merged at 20:39 UTC, for the LFM reasoning-channel fix.

Video input needs semantics, not clever adjacency

The Qwen-VL change adds support for a “frame merge” behavior in llama.cpp’s multimodal path. The PR explains that Qwen-VL-based models have a conv3d path for image input that can merge two same-size input images into the same output embeddings. In the intended case — adjacent frames from video-like input — that can reduce decoding cost and memory pressure. In local inference, where developers are trying to squeeze multimodal models onto real hardware instead of infinite demo clusters, those savings matter.

The implementation touches the right files for this kind of change: tools/mtmd/models/qwen2vl.cpp, tools/mtmd/models/qwen3vl.cpp, tools/mtmd/clip.cpp, tools/mtmd/mtmd-image.cpp, and related model headers. The PR also states the limit clearly: this is part of broader video-input work, not a complete end-to-end video pipeline, and it does not add a new public API. The merge behavior is automatic inside the mtmd path for adjacent same-size images under the relevant conditions.

The reviewer discussion is the important part. Maintainer review pushed back on the assumption that two adjacent images should be treated as mergeable frames. Two images can be adjacent because they are temporally related, or because a user uploaded a handbag photo followed by an eBay listing screenshot, or because an agent attached a UI screenshot next to a diagram. Those are different semantic objects. Merge them blindly and the model may lose the structure it needs to answer correctly.

The PR author’s response was that llama-server inserts a newline between two images, so ordinary server inputs should not be accidentally merged, and the merge path is aimed at custom video helper formatting. That is reassuring, but it also proves the larger point: the OpenAI-compatible chat shape is a weak container for video. It can carry images. It does not, by itself, say which images are frames, which are unrelated attachments, which belong to the same clip, or which should remain separate evidence. Video-capable local inference needs explicit temporal semantics. Otherwise, every optimization becomes a guess.

Practitioners should take this as a design warning. If you are building multimodal agents on local runtimes, do not rely on attachment order as your only meaning layer. Mark frames as frames. Mark separate artifacts as separate artifacts. Test the exact runtime behavior with multiple images: two video frames, two unrelated screenshots, screenshots plus diagrams, and mixed sizes. An optimization that is correct for video can be wrong for document analysis or UI automation.

The reasoning channel is now part of the API contract

The LFM2/LFM2.5 fix is less visually exciting and probably more urgent for many agent builders. PR #24234 fixes a case where LFM2/LFM2.5 models copy reasoning_content into thinking, and because LFM2.5-8B-A1B is always a reasoning model whose chat template does not expose a switch to disable reasoning, thinking could leak into ordinary content when reasoning was disabled with -rea off. The fix drops thinking content in that disabled path, similar to DeepSeek handling, and adds tests around the template behavior.

That sounds like an edge case until you remember what agent systems do with content. They log it. They paste it into tickets. They put it in PR comments. They feed it to downstream parsers. They hand it to tools. They sometimes show it to customers. A stray <think> block is funny once in a terminal. In a production workflow, it is a leakage bug, a parser bug, a compliance issue, or all three at once.

Reasoning models have made the runtime contract messier. There is user-visible answer text, hidden or semi-hidden reasoning content, tool-call JSON, function arguments, system traces, and sometimes vendor-specific thinking fields. Managed APIs hide much of that complexity behind policy and schema. Local stacks inherit it directly. If the chat template is wrong, the model may appear to work while returning content in the wrong channel. That is worse than a load failure because it creates false confidence.

The practical lesson is to pin and test the inference runtime the same way you pin the model. “It loaded the GGUF” is not a readiness check. Add regression tests for the chat templates you actually use. Test reasoning enabled and disabled. Test tool-call formatting. Test multi-turn conversations where prior reasoning content might round-trip. Test the exact model families in deployment, because DeepSeek, Qwen, LFM, Llama, and other lines do not all use the same template assumptions. If your local agent writes to GitHub, Slack, Jira, email, or customer support systems, treat channel separation as a safety property.

Local model economics moved from tokens to operations

This is the part local-AI boosters tend to undersell. Local Qwen stacks, NVFP4 quantization, vLLM support, Ollama workflows, and llama.cpp builds make the marginal-token story attractive. Privacy can be better. Cloud invoices can fall. Latency can improve for some workloads. But the operational cost does not disappear. It moves into runtime compatibility, GPU memory tuning, quantization behavior, multimodal preprocessing, and template correctness.

NVIDIA’s Qwen3.6-35B-A3B-NVFP4 model card is a useful context signal here: local and semi-local stacks are moving fast, with MoE transformer blocks quantized and vLLM readiness advertised. That kind of model-card momentum is real. But model cards do not guarantee your runtime understands a vendor’s multimodal path or reasoning template. The runtime is not a dumb loader anymore. It is part of the model behavior.

For engineering teams evaluating local coding agents or multimodal assistants, the action item is boring and valuable: maintain a compatibility matrix. Model version, quantization, runtime version, chat template, reasoning support, tool-call support, image/video handling, GPU offload settings, and known failure cases. Run smoke tests before upgrading. Keep fixtures for the weird cases, not just the happy path. Two unrelated images should remain two unrelated images. Reasoning-off should mean reasoning-off in the emitted content. Tool calls should parse after a long multi-turn session, not just in a one-shot demo.

The June 6 llama.cpp changes are small only if you judge importance by launch-event theatrics. In practice, they show the stack maturing in the places that break real deployments: multimodal semantics and channel hygiene. The model weights get the headlines. The serving layer decides whether users can trust the system.

The editorial take: local agents are becoming infrastructure, and infrastructure is mostly plumbing. If you do not track runtime releases with the same seriousness as model releases, you are not running local AI. You are running a science project with better branding.

Sources: llama.cpp GitHub releases, Qwen-VL frame-merge PR, LFM reasoning leak PR, NVIDIA Qwen3.6 NVFP4 model card, Liquid AI LFM2.5 model card