LiteLLM 1.89 RC Turns MCP and A2A From Connectors Into Governed Agent Infrastructure

LiteLLM’s v1.89.0-rc.1 looks, at first pass, like another gateway release with the expected parade of provider additions and integration fixes. That is the wrong read. The important story is that LiteLLM is turning MCP and A2A from “connectors my agent can call” into governed infrastructure: signed images, OAuth passthrough, issuer-scoped JWT validation, per-MCP-server rate limits, OpenTelemetry spans, blocked-call guardrail traces, per-server environment variables, and new agent gateway providers for LangFlow and watsonx Orchestrate.

That is not a feature grab bag. It is a fairly direct answer to the problem every serious agent deployment eventually hits: the risky unit is no longer just the model call. It is the tool call, the delegated agent call, the forwarded credential, the hidden parameter, the session bridge, the container image, and the trace that either proves what happened or leaves operators guessing.

The gateway is becoming the policy plane

The release was published on June 6 as v1.89.0-rc.1, with LiteLLM’s repository sitting around 49,631 stars, 8,688 forks, and 3,482 open issues during the research window. The numbers matter less than the shape of the work. PR #28356, which adds MCP OAuth passthrough and issuer-scoped JWT auth, was not a quick patch: 135 commits, 42 files changed, 4,924 additions, 413 deletions, 63 comments, and 75 review comments. That is what “auth plumbing” looks like when it is actually infrastructure.

The release also leads with Docker image signature verification. LiteLLM says its Docker images are now signed with cosign using a key introduced in commit 0112e53046018d726492c814b3644b7d376029d0, and recommends verifying against the pinned commit hash rather than trusting only the release tag. That is the correct level of paranoia for an agent gateway. If this process sits between your agents and internal tools, the container image is part of your tool-permission boundary. Treating it like a generic Python service image is how supply-chain incidents get promoted to “agent incidents” with better branding.

The MCP documentation describes LiteLLM Proxy as an MCP Gateway with fixed endpoints for tool listing, tool calling, prompts, and resources across Streamable HTTP, SSE, and stdio transports. It also names the governance surface directly: permission management by key, team, and organization. The MCP control docs go further, with a five-level permission model — key, team, end user, agent, and organization — where permissions intersect and the organization acts as a ceiling. Translation: if configured correctly, access does not depend on a single API key floating around a workflow. It composes through identity layers.

Rate limits belong at the tool boundary

One of the most practical changes is PR #29482: per-MCP-server requests-per-minute limits for keys and teams. It is only 499 additions across 12 files, but it points at a bigger architectural shift. Model-level rate limits are not enough once agents can call tools. The expensive thing might be a GitHub search, a warehouse query, a Slack export, a browser action, a ticket mutation, or an internal API that was never designed for autonomous callers.

Teams have been using model budgets as a proxy for agent budgets because the model invoice is easy to see. That misses the more dangerous spend and blast-radius paths. A runaway agent that burns tokens is annoying; a runaway agent that hammers an MCP server connected to production systems is an incident. Per-server RPM limits let operators express the real boundary: this key, this team, this tool server, this allowed rate. It also gives security and platform teams something concrete to tune before they need a postmortem.

The same logic applies to tool and parameter controls. LiteLLM’s MCP permission docs support allowed_tools, disallowed_tools, and allowed_params. If both allowed and disallowed tools are set, the allowlist wins. Requests with disallowed parameters get rejected with a 403. That sounds bureaucratic until you remember that many tools are only dangerous when called with the wrong argument: a broad date range, a destructive flag, a private repo selector, a high limit, a production environment name. Tool governance that stops at “can call server X” is too coarse for real systems.

OAuth passthrough is useful because it is dangerous

The release’s sharpest edge is credential flow. The auth reference says LiteLLM exposes both MCP Gateway and A2A Agent Gateway surfaces that share caller authentication and identity headers, but diverge in backend authentication. MCP has an explicit auth_type enum: no auth, static API keys and bearer tokens, OAuth2 with PKCE or machine-to-machine flows, RFC 8693 on-behalf-of token exchange, and AWS SigV4. A2A, by contrast, infers outbound auth from provider parameters.

That split matters. OAuth passthrough, OBO flows, per-user forwarding, and stored tokens are exactly what make agent gateways usable inside companies. They are also exactly where teams accidentally destroy auditability. The docs recommend using x-litellm-api-key when the inbound Authorization header carries a different token for OAuth passthrough, OBO, or A2A per-user forwarding. That is not a naming preference. It is a warning label.

If a gateway confuses “the token that authenticates the caller to LiteLLM” with “the token that should be forwarded to GitHub, Slack, an MCP backend, or an agent provider,” your logs become fiction. You may still get a 200 response, which is the worst kind of failure. Builders should diagram every credential hop before enabling this in production: human or client identity, LiteLLM virtual key, end-user header, agent ID, MCP server auth, upstream service token, and refresh/persistence behavior. If that diagram contains “then the gateway handles auth,” you have not finished the design.

PR #28356’s review size suggests the maintainers understand this. The summary calls out authentication and authorization changes for MCP, including anonymous cold-start bypasses, header stripping, 401 propagation, and JWT validation. Those are not release-note decorations. They are the corners where gateway products either become dependable infrastructure or quiet privilege-escalation machines.

Invisible tool calls are operational debt

Observability is the other half of the release. PR #29468 adds MCP semantic conventions to OpenTelemetry v2, with closed call_mcp_tool requests emitting a client span named tools/call {tool}. The PR says MCP tool calls previously flowed through the proxy without producing spans. That is exactly the kind of absence that feels fine during a demo and ruins your week in production.

PR #29470 adds guardrail spans when a passthrough guardrail blocks a request. This is more important than it sounds. Many systems trace happy paths and lose policy-denied paths, which means the moment your controls do their job, the evidence disappears. A blocked tool call should be as visible as an allowed one: caller, tool, server, reason, trace ID, policy result, and downstream effect. Otherwise, security gets anecdotes and platform gets vibes.

The A2A work points in the same direction. PR #28963 adds LangFlow as an A2A agent provider, including POST /api/v1/run/{flow_id} support and bridging A2A contextId to LangFlow session_id for multi-turn conversations. PR #29410 adds watsonx Orchestrate with auth, submit-run, polling, SSE, and fallback streaming. Once LiteLLM speaks both MCP and A2A, it is no longer only a model proxy. It becomes a routing layer for tools and agents, where spend attribution, session continuity, policy, and trace propagation meet.

That is powerful. It is also a centralization bet. The upside is one place to govern identities, tools, agent calls, budgets, and traces. The downside is one place that must be upgraded, reviewed, monitored, and threat-modeled like infrastructure. “We put it behind the agent gateway” should eventually mean the same thing as “we put it behind the API gateway”: policy, logs, quotas, auth, versioning, and incident procedures. Not just a convenient URL.

What engineering teams should do next

If you run LiteLLM today, the practical checklist is straightforward. First, verify images with cosign before promoting this release candidate anywhere near production. Second, inventory MCP servers by risk: read-only knowledge tools, user-scoped SaaS tools, expensive APIs, state-changing internal systems, and anything with production data. Third, apply per-server RPM limits where the backend cost or blast radius is not captured by model spend.

Fourth, stop treating tool permissions as binary. Use allowlists for sensitive servers, block dangerous tools where needed, and restrict parameters for operations where one argument changes the threat model. Fifth, test denied paths deliberately. A blocked passthrough call should emit a useful span and an understandable error, not vanish into a generic failure. Sixth, document credential flow with enough specificity that an incident responder can answer: whose identity made this call, which token reached the backend, what policy allowed it, and where the trace lives?

The broader takeaway is that agent infrastructure is maturing in the least glamorous place: the gateway. Everyone wants to compare agents by model quality, benchmark wins, and slick IDE demos. The real production question is duller and more important: can you govern the things the agent touches after the model decides what to do?

LiteLLM 1.89 RC is a useful marker because it says the quiet part clearly. The winning agent stack is not the one with the longest connector list. It is the one that can make identities, tool calls, tokens, rates, denials, and traces explicit enough that operators do not have to trust the agent’s summary of its own behavior. Good. Agents already hallucinate text; we do not need them hallucinating audit trails too.

Sources: LiteLLM GitHub release, LiteLLM MCP docs, LiteLLM MCP permission docs, LiteLLM gateway auth reference