, AI is now everywhere. But for most organizations, it’s delivering a very small fraction of its potential. There are multiple studies, e.g. from BCG and MIT, that say over 80% of AI initiatives are failing. But this isn’t new. It also happened in the era of business intelligence, big data, data science, analytics, and machine learning. There were studies that pegged failure rates at 80-90%. That number isn’t likely to change. What leaders should be thinking about is how to be among the 10-20% winners.
Why do most organizations fail? Because they are pulled in a thousand directions at once. Vendors sell point solutions that solve only a sliver of the problem. Consultants push frameworks that promise a lot but deliver in fragments. Startups sell innovation, but disconnected from the larger picture. The result: organizations are left with pilot projects, chatbots, and narrowly scoped initiatives that never aggregate into any meaningful business outcomes. This is happening again: there is real urgency around AI, but without focus, one ends up in the 80%.
The path out of this chaos is to focus and build capabilities that lift many boats in the company. Enterprises need to build around three foundational pillars. And if done well everything else will fall into place.
Pillar 1: Build a Context Platform — The Fabric of Enterprise Truth
It is no secret that one needs high-quality data to derive the most benefit out of AI. but most organizations are still thinking about it within the constructs of the previous era. Having a unified data platform, having a single source of data truth, having golden tables, data governance, etc. All of these are needed. But what AI needs today is not just data but context.
Most enterprises still do not have a single version of truth. This in a way can be an advantage today. As a late mover one can build a context platform instead of a pure data platform. It will be not just about having data that can be joined to each other and queried together. Instead the Context Platform is about providing the full context around the data.
For example, to understand the next best action for a customer, advanced AI reasoning systems benefit deeply by having full situational awareness. This means giving traditional metrics such as revenue and product usage, but also providing rich context. E.g.:
- Customer engagement context: Every email, every support ticket, every interaction across the organization, etc.
- Business context: Renewal information, contract terms, and previous pricing actions, etc.
- Market and industry context: Competitor activity, regulatory changes, industry trends, macroeconomic factors, etc.
All of this situational awareness can greatly enhance the recommendations. Let us say one is setting up a cross-sell engine. Without a context fabric, a recommendation might be based solely on current revenue and product usage patterns. But with a context fabric, the AI can combine usage data with past interactions and market signals such as industry news. And then one might find that the logical next best product is the one that the customer already showed resistance to in past sales conversations. However the customer is facing competitive threats that make another product much more suitable.
But how to build such a fabric? A context platform is an evolution of a data platform and it integrates multiple components:
- Connected and indexed data: Batch and stream ingestion from apps, SaaS systems, data lakes, and operational systems
- Semantic enrichment: Extraction of entities and relationships into knowledge graphs, enriched with ontologies, lineage, and business glossaries
- Hybrid retrieval capability: Multi-modal search combining keyword, vector, and graph traversal. Reranking to ensure relevance for AI models
- Governance: User level access control, PII redaction/masking, audit trails, and AI governance workflows
- Evaluation and observability: Infrastructure for continuous monitoring of relevance, answer accuracy, latency, and cost
On the technical front, there are multiple ways to build this platform. All large cloud platforms provide products that can be stitched together for this purpose. We will discuss one such stack later.
Pillar 2: AgentOS — Agent Operations Platform
With the context fabric in place, the next pillar is to enable the organization with the right intelligence tooling. Even with a context fabric, most organizations will remain stuck in POC purgatory. The root cause is fragmentation: dozens of chatbots, hundreds of pilot projects, but no enterprise capabilities to help build in a coherent way, at scale.
The AgentOS is a platform that enables a large number of employees to use and build their own AI agents in a governed way. AI agents that help them increase their efficiency and automate their tasks. But it cannot be just about improving efficiency. The platform should enable technical teams to build out ambient agents that run in the background and not only automate but do large portions of current tasks in an automated way, and pull in humans in the loop for exception and error management. This governed, reusable, runtime platform to build, deploy, monitor, and secure AI agents at scale has 3 core services it provides:
- Co-pilots: Integrated directly into the right tools and workflows, enabling real-time assistance and decision-making.
- Agent-building frameworks: GUI based tools and pro-code SDKs that allow teams to rapidly create domain-specific agents on top of the context fabric.
- Ambient agents: Operate in the background, autonomously handling routine tasks while humans manage exceptions.
There are 6 set of capabilities that AgentOS should aim to provide in its end state:
- Build: GUI and pro-code agent creation with multi-agent orchestration
- Ground: Connectors, RAG retrieval, long- and short-term memory
- Act: Secure API and tool access, workflow actions, MCP support
- Interoperate: Open protocols for cross-agent communication, avoiding vendor lock-in
- Trust: RBAC, audit trails, identity management, content safety
- Monitor: Dashboards for command, cost, quality, and safety metrics.
This of course is a very advanced set of capabilities. But not everything needs to be built at once, nor all the components are needed to start. One should start with 2-5 user teams and build around their needs, have the steel threads, and then expand. Again, this is possible with multiple vendor stacks. Below is one example of a largely open source stack that brings together both the Context Fabric and AgentOS.
Without an orchestration layer, every agent is just another silo. With it, they become an interconnected force multiplier.
Pillar 3: Workforce Magic
Even the best technology stacks fail without human adoption. And specially with AI, human in the loop is a critical component. Research from McKinsey projects that 60-70% of today’s work activities will be automated by 2030. WEF estimates 78 million new jobs will emerge even as 92 million are displaced. All of this implies that the fundamental nature of work will change. The organizations that prepare their workforce for this will be able to leverage AI better. This will prepare employees for changes to come. And prepare employers to be the ones who are the successful 20%
Employees don’t need to just learn to use AI. They need to redesign their workflows, make judgment calls, and optimize agent-human collaboration e.g. with Ambient Agents.
A structured workforce program can have three components:
- Skill passports: Map each role to concrete AI-era competencies and train the workforce.
- Agent Builder Sprints: Teach and empower staff to build agents on approved infrastructure and own their efficiency goals.
- AI re-write: Make leaders responsible for making their orgs AI Native. Track redeployed hours, AI fluency, and processes redesigned. Not just cost savings.
AI success will be inseparable from workforce readiness. Technology, agents, and context matter. But they all still need humans to operate effectively. Without that, enterprise AI fails.
Conclusion
I see many organizations paralyzed by this fast-moving technology. Knowing that they need to move, but unable to do so at the needed pace. While there are many things one can do, doing the above will build a durable advantage that not only helps the enterprises succeed, but also helps employees become partnera in that success and unleash AI at scale.
Shreshth Sharma is a Business Strategy, Operations, and Data executive with 15 years of leadership and execution experience across management consulting (Expert PL at BCG), media and entertainment (VP at Sony Pictures), and technology (Sr Director at Twilio) industries. You can follow him here on LinkedIn.