SoftSkill Compresses Agent Skills Into 32 Virtual Tokens — and Shows Where That Trick Breaks

Markdown skills are one of the more sensible things agent platforms have converged on. They are readable, reviewable, portable, and cheap to author. They are also long, repeatedly injected, and interpreted from scratch by a model that may or may not translate prose into the behavior you intended. The obvious compiler question was coming: can the useful behavior be compressed into something much smaller?

SoftSkill answers with a qualified yes. The paper proposes compressing natural-language skills into a compact continuous prefix while keeping the base model frozen. It initializes from a human-readable skill, tunes a soft delta with next-token prediction, and deploys a length-32 latent behavioral prior instead of hundreds or thousands of hard prompt tokens. For narrow single-round tasks, the results are genuinely interesting. For long-horizon agentic work, the caveat is doing a lot of load-bearing work.

That caveat is not a reason to dismiss the paper. It is the reason the paper is useful. Compressing answer behavior is not the same as compressing procedural judgment. If teams remember that distinction, SoftSkill points toward a practical future where readable skills remain the source of truth, compiled skills carry the runtime load, and regression tests decide whether the compiled behavior still deserves to ship.

A 32-token prefix is a tempting invoice reducer

The main experiments use Qwen3.5-4B with a length-32 SoftSkill prefix. In the primary single-round setting, SoftSkill improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA. Relative to Microsoft’s SkillOpt baseline, SoftSkill improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath.

The compression ratio is the product hook. The paper replaces Markdown skill artifacts of 2,035 tokens for SearchQA, 671 tokens for LiveMath, and 407 tokens for DocVQA with a 32-token soft prefix. In reproduced compression-side results, SearchQA moves from SkillOpt’s reproduced 72.0 to SoftSkill’s 76.4, while average generated tokens drop from 98.4 to 8.67. LiveMath is the dramatic case: reproduced SkillOpt 37.1 to SoftSkill 60.5, with average generated tokens dropping from 4,422.0 to 5.99.

That is not a rounding error. If your agent product injects the same long skill over and over, a learned prefix that preserves behavior can reduce latency, context pressure, and token spend. It can also make small models more useful by giving them a task-specific behavioral prior without touching base weights. For low-risk repetitive workflows, this is exactly the kind of optimization that separates a demo from a product with margins.

DocVQA is the reminder that compression is not universally better. Reproduced SkillOpt scores 90.1 versus SoftSkill’s 89.3, though SoftSkill still reduces output tokens from 29.6 to 11.1. Model-size sweeps are also non-monotonic: Qwen3.6-35B-A3B improves SearchQA and DocVQA but performs less cleanly on LiveMath than Qwen3.5-9B in some placements. Bigger model plus soft prefix is not automatically a monotonic ladder to heaven. It is an artifact that needs evaluation.

The unreadable skill blob problem

The governance tradeoff is immediate. Natural-language skills are inspectable. A reviewer can ask whether an instruction is too broad, unsafe, stale, or inconsistent with policy. A soft prefix is behavior without legibility. That may be fine for a SearchQA helper. It is much less obviously fine for an agent that edits files, approves transactions, operates browsers, touches infrastructure, or handles user data.

This is where the compiler analogy helps. Nobody serious treats a compiled binary as the only source of truth. The source code still matters, the build process matters, the tests matter, and the artifact is versioned. Soft skills should get the same treatment. Keep the Markdown skill as the reviewed source of truth. Compile it into a soft prefix for runtime efficiency. Version both. Run regression tests that compare hard-skill and soft-skill behavior. Reject compiled prefixes that preserve benchmark accuracy while dropping safety constraints.

The paper’s own boundary statement is the part agent teams should quote carefully: agentic execution remains harder. Sparse trajectory imitation provides signal, but does not robustly compress long-horizon procedural behavior yet. That is exactly what one should expect. Long-horizon agents need to know when to inspect state, when to call tools, how to recover from failed commands, how to preserve user constraints, and when to ask for human approval. Those are not just answer patterns. They are policies across time.

For practitioners, the action item is to classify skills before compressing them. Low-risk, repetitive, answer-shaped skills are good candidates. High-risk procedural skills should remain readable and auditable unless compression is proven to preserve both capability and constraints. If a skill grants tool-use behavior, controls data handling, or encodes safety boundaries, do not replace it with latent goo because a benchmark number looked nice. Make the test suite earn that trust.

There is also a product-design angle. Users and administrators increasingly want to know why an agent behaves a certain way. Human-readable skills give them something to inspect and edit. SoftSkill-style prefixes could become an implementation detail, but they should not become the only interface. The right UX may be “write/review the skill in Markdown, compile for efficiency, show test coverage, allow rollback.” Boring. Also probably correct.

The editorial take: SoftSkill is a promising compiler story for agent skills, not a license to ship unreadable behavior blobs everywhere. Compress the parts that are cheap to verify. Keep the parts that require trust in daylight.

Sources: arXiv, SoftSkill code, Microsoft SkillOpt, Anthropic agent skills, LoRA