Microsoft 365 Copilot’s Usage Report Finally Admits Agents Need Their Own Adoption Metrics
Microsoft’s updated Copilot usage-report documentation is not a flashy AI announcement, which is exactly why it is worth paying attention to. The company is now foregrounding agent usage alongside the usual Microsoft 365 Copilot adoption numbers: enabled users, active users, prompt volume, app-level usage, Copilot Chat split between work and web, and active users of agents built by the organization. That is admin-console plumbing. It is also the moment agents start being treated like enterprise software instead of demo theater.
The metric to watch is “Active agent users.” Microsoft defines it as unique Microsoft 365 Copilot users in the organization who used agents built by the organization, including admin-approved agents and agents created through Agent Builder and shared internally. That last clause matters. Microsoft is not only counting centrally built platform-team agents; it is also acknowledging the messy middle where business units and internal teams create agents, share them, and expect the tenant to make sense of adoption later.
The report is available over the last 7, 30, 90, or 180 days, and Microsoft says usage data usually appears within 72 hours after the end of a UTC day. Agent usage is available starting November 1, 2024, but with a major limitation: the current reporting is limited to agents built by the organization. Microsoft- and partner-built agent usage is expected later. That makes the report useful, but not complete. Admins who read it as a full map of agent activity will undercount the surface area.
Prompt counts were never enough
Copilot adoption has always had a measurement problem. Licenses are easy to count and easy to oversell. Prompt counts are better, but still crude. A user can submit fifty prompts because Copilot failed to understand the task, or two prompts because it solved the problem cleanly. Raw usage does not equal value. Agents are a different signal because they represent packaged workflows: procurement intake, HR policy lookup, IT triage, sales-account prep, contract review, onboarding, reporting, or whatever else the organization has decided to automate inside Copilot.
That is why active agent users is a meaningful step. If a custom procurement agent has repeated use from the buyers it was designed for, that suggests a workflow is becoming real. If an HR policy agent spikes after launch and then disappears, that suggests novelty rather than adoption. If a custom engine agent with broad action permissions suddenly becomes popular outside its intended audience, that is not just “engagement.” It is a risk review. Metrics do not govern anything by themselves, but they tell you where to look.
The distinction between declarative agents and custom engine agents is important here. Microsoft’s extensibility docs describe declarative agents as agents that use Copilot’s orchestrator and models, with custom instructions, Microsoft 365 knowledge sources such as SharePoint and OneDrive, connector knowledge, and custom actions. Custom engine agents bring their own orchestrator, model choices, hosting model, autonomy, and compliance burden. Adoption reporting should not treat those two categories as equivalent. A declarative FAQ agent and a custom engine agent that can call business systems are not the same operational risk because they happen to share the word “agent.”
The Viva Copilot Dashboard context adds another product-policy wrinkle. Tenants with at least 50 Copilot licenses or at least 50 Viva Insights licenses get richer agent-related insights, Agent Dashboard features, benchmarks, intelligent summaries, scoped group-level data, delegation support, and survey data where available. Smaller pilots may not have the same reporting depth. That creates a practical problem: teams are often told to pilot agents before broad rollout, but the best dashboarding may arrive only once the rollout is already substantial. If the pilot matters, instrument it separately. Do not wait for the tenant-level dashboard to become your only source of truth.
Agents need product management, not launch videos
The operational takeaway for builders is simple: ship internal agents like products. Define the target users. Define the job to be done. Define first-use success. Define repeated-use expectations. Define what failure looks like and where users escalate. Decide whether the agent should be retired if usage stays low after a certain window. The presence of active-user reporting should push teams away from “we made a cool bot” and toward “we own a workflow with adoption, support, telemetry, and a budget.”
For admins, the checklist grows. Track who built the agent, who approved it, what knowledge sources it can access, what actions it can take, whether it is declarative or custom-engine, and which users are expected to use it. Then compare actual usage against intent. High usage in the intended audience may validate the investment. High usage outside the expected audience may mean the agent is solving a broader problem, or that permissions are too loose. Low usage may mean the agent is bad, undiscoverable, unnecessary, or solving a problem people do not actually have.
There is a finance angle too. Copilot rollouts are expensive enough that organizations eventually ask for ROI evidence. Agent metrics can help, but only if paired with business outcomes. “Active users” is not the finish line. The useful questions are more concrete: did the IT triage agent reduce ticket handle time? Did the sales prep agent shorten account research? Did the contract agent reduce review cycles? Did the HR agent lower repetitive policy questions? If the answer is unknown, the metric is decorative.
The limitation around Microsoft- and partner-built agents should also shape governance. If the current report only covers agents built by the organization, then tenant owners need separate inventory and policy controls for the rest of the ecosystem. Otherwise, leadership may see a clean internal-agent chart while external or partner agents are doing meaningful work elsewhere. Reporting lag is not permission lag. The agent surface exists whether the dashboard has caught up or not.
The broader direction is healthy. Microsoft is slowly moving agents into the normal enterprise-software lifecycle: build paths, admin approval, usage reporting, dashboarding, benchmarking, and, eventually, sharper accountability. That is where this category needs to go. The industry has spent too long treating agents like magic coworkers. Coworkers have managers. Software has owners. Agents need both, plus telemetry.
The forward-looking take: active agent users is an imperfect metric, but it is the right kind of imperfect. It forces the conversation away from “how many Copilot licenses did we buy?” and toward “which agent-backed workflows are actually being used, by whom, and with what risk?” That is the conversation enterprises should have been having before half these bots showed up in production. Better late than unlogged.
Sources: Microsoft Learn, Microsoft Viva Copilot Dashboard docs, Microsoft 365 Copilot agents overview, Agent Builder docs