3 minutes read

Context makes the Coworker: centralized context preferred ~2.5x as often as off-the-shelf MCP tools, which consumed 30% more tokens

  • Neil DhruvaEngineering
  • Carlton KozeyApplied Scientist
  • Kari StehrSoftware Engineer
  • Julie MillsPMM

The promise of AI coworkers is that employees can get more done when AI is connected to their external data sources, including Google Docs, Gmail, Google Calendar, Salesforce, and Atlassian. That connection enables large-scale data analysis, smarter inbox and calendar management, faster content creation, and better information retrieval. However, realizing that promise requires AI whose output an employee can take and use almost as-is at a token cost commensurate with the task.

We wanted to understand how the context layer shapes the effectiveness and efficiency of AI coworkers to generate these ‘take and use’ results. It is well established that the quality of the underlying context, the indexes and knowledge graphs, drives response accuracy. What is less understood is the economic tradeoff of indexes with better relevance and ranking.

To prove the value of the context layer, we benchmarked a centralized index against off-the-shelf MCP tools across ~175 queries. We found that:

  • The centralized index was preferred ~2.5x as often as off-the-shelf MCP tools

  • Off-the-shelf MCP tools consumed ~30% more tokens

The federated search token tax

The Model Context Protocol (MCP) is an open standard that lets AI models connect to external tools to access data and take action through a consistent interface. MCP standardizes tool connectivity, giving models a consistent way to call external systems. However, MCP does not standardize the quality of the underlying tools themselves.

There have traditionally been two approaches to solving for enterprise context: federation and indexing. Federation relies on the search provided out-of-the-box by each individual connector. Centralized indexing ingests and normalizes data from across all sources into a single layer.

There are several well-understood challenges with adopting a federated-only approach:

  • The quality of the underlying search varies significantly, with some tools supporting only lexical or semantic retrieval but not both.

  • The number of tool calls required to gather the data increases, as each tool has to be called individually.

  • Access to data is limited to user-specific signals, which means you lose visibility into how that data is used across the wider enterprise.

  • Latency compounds quickly when you are dealing with multiple API calls to different tools.

Federated search often compensates by having AI over-fetch data across multiple tool calls and rely on multiple reasoning loops to synthesize the results. Both are costly workarounds in terms of latency and token usage, and they still tend to produce inaccurate responses.

Evaluation design

We standardized the harness and compared a centralized index against the off-the-shelf MCP servers, grading every response on utility, correctness, completeness and execution quality.

3 queries showcasing the power of context

What deployments had SSAT alerts last week?

Exhaustive data analysis across all affected customers requires going broad before drilling down. Here is how both setups answered.

What deployments had SSAT alerts last week?

Cognition

Based on the search results, here's what I found for last week's SSAT alerts (week of 4/28–5/4):

  • Maxium Corp — Google Drive indexed docs spiked steeply, with a corresponding spike in incidence in top 10 results.

  • Ironbridge Health and Provo Systems — Both dropped for a day but recovered on their own.

Want me to pull up more detail on any of these, like the linked Slack threads?

Off-the-shelf MCP

The #ssat-alerts channel had alerts most days last week. Let me dig into that thread and the others to find which specific deployments were flagged.

  • Maxium Corp prod — was the deployment flagged on SSAT alerts, appearing across all 5 daily alert CSVs.

This response was slightly altered and pseudonyms were used to protect the names of customers and sensitive data.

While off-the-shelf MCP was able to dig deep into a specific alert for one customer, it failed to first go broad and identify the full set of customers affected by the alert before beginning troubleshooting.

Find the most recent customer feedback themes and summarize the top 5

The centralized index synthesized data across sources to identify the top customer asks. Off-the-shelf MCP tools surfaced product requests that were unrelated to the question and fell short on completeness.

Context in the Cowork Era

Centralized indexes and knowledge graphs outperformed off-the-shelf MCP servers on both quality and efficiency. As enterprises scale AI across longer-running, more complex work during a period of rapidly rising frontier model costs, the design of the context layer becomes a direct input to both the quality and the economics of AI at work.

Back to all stories

Turn complexity into confident action.

Request a Demo