The rapid adoption of AI, particularly agentic AI, is transforming enterprises into copilots, assistants, and autonomous task-runners, with nearly two-thirds of companies experimenting with AI agents by late 2025, a significant increase from previous years, according to McKinsey’s annual AI report. While initial AI pilots often demonstrate success, a striking reality emerges: only one in ten companies manage to scale their AI agents effectively. This bottleneck isn’t typically due to limitations in AI models themselves, but rather a fundamental deficiency in the data architectures that underpin them, hindering the reliable delivery of essential business context to both humans and AI agents. Experts emphasize that the coming months and years are critical for businesses to establish the right data architecture, as the unpredictable evolution of AI necessitates a robust foundation. Irfan Khan, president and chief product officer of SAP Data & Analytics, underscores this point, stating, "The only prediction anybody can reliably make is that we don’t know what’s going to happen in the years, months — or even weeks — ahead with AI. To be able to get quick wins right now, you need to adopt an AI mindset and … ground your AI models with reliable data." In the age of AI, data’s importance transcends its traditional role, with the effectiveness of agentic AI now more heavily dictated by the soundness of enterprise data architecture and governance than by the models themselves. To achieve scalability, businesses must embrace modern data infrastructures that deliver data enriched with context.

A pivotal shift in the approach to data for AI agents lies in recognizing that "more business context, not necessarily more data" is the key differentiator. Traditional data management often prioritizes structured data over unstructured, assigning it higher value. However, AI challenges this paradigm. For AI agents, high-value data is less about its format and more about its embedded business context. Critical business functions like supply-chain operations and financial planning are inherently context-dependent. While granular, high-volume data streams, such as IoT, logs, and telemetry, can offer significant value, their true potential is unlocked only when presented with relevant business context. Khan asserts that the primary risk for agentic AI is not a scarcity of data, but a lack of grounding. "Anything that is business contextual will, by definition, give you greater value and greater levels of reliability of the business outcome," he explains. "It’s not as simple as saying high-value data is structured data and low-value data is where you have lots of repetition – both can have huge value in the right hands, and that’s what’s different about AI." This essential context can be cultivated through integration with software, on-site analysis and enrichment, or robust governance pipelines. Data that lacks these qualities is likely to be untrusted, a sentiment echoed by the Institute for Data and Enterprise AI (IDEA), which found that two-thirds of business leaders do not fully trust their data, contributing to a "trust debt" that impedes AI readiness. Overcoming this deficit requires the establishment of shared definitions, semantic consistency, and reliable operational context to align data with its true business meaning.

The pervasive "data sprawl," a consequence of the decade-long trend of separating compute and storage and embracing cloud-scale flexibility, presents a significant challenge for AI adoption. This architectural shift has led to data being dispersed across multiple clouds, data lakes, warehouses, and a multitude of SaaS applications. As companies pivot towards AI, this sprawl intensifies. More than two-thirds of companies identify data silos as a primary obstacle to AI adoption, with over half of enterprises grappling with a thousand or more data sources. While the previous era focused on building foundational platforms like SaaS and data lakes by separating compute and storage, the current era demands the delivery of the right data to autonomous AI agents tasked with diverse business functions. Khan highlights that the separation of compute and storage was a major innovation in data management, but the critical differentiator now is the harmonization and extraction of value from data across multiple content sources. This necessitates a semantic or knowledge layer that operates across platforms, encapsulates business rules and relationships, provides a business-contextual and governed view of data, and enables both humans and AI agents to access information appropriately. Legacy data architectures are ill-equipped to power the autonomous AI systems of the future, as indicated by Deloitte’s State of AI in the Enterprise report, which found that only four in ten companies believe their data management processes are ready for AI, a decline from the previous year. This realization highlights the growing awareness of infrastructure shortcomings as businesses explore AI deployment.

Contrary to speculation by some investors and technologists that AI agents will render SaaS applications obsolete, Khan strongly refutes this notion. He posits that value has consistently ascended the technology stack over the past 15 years, moving from on-premises infrastructure to IaaS, PaaS, and then SaaS. Agentic AI represents the next evolutionary layer, providing an interface for accessing data and interacting with business logic. While value rises, foundational layers do not disappear. "SaaS doesn’t go away," Khan asserts. "It just means SaaS and these agents will cooperate with one another. Companies are not going to throw away their entire general ledger and replace it with an agent. What’s the agent going to do? It doesn’t know anything without business context and business processing." In this evolving landscape, the software stack is being reconfigured so that applications and data provide governed context within which AI can operate effectively. SaaS applications retain their role as systems of record, while the semantic layer emerges as the business-context source of truth. AI agents function as a new engagement layer, orchestrating across systems, and both humans and agents are recognized as "first-class citizens" in their access to business logic. Critically, Khan warns that agents cannot directly interface with every operational system, stating, "If we’re saying agents are going to take over the world… you can’t have an agent talking to every operational backend system. It just doesn’t work that way." This reality further amplifies the importance of a semantic or business-fabric layer.

For enterprises seeking to embark on this journey, the starting point lies within their existing data platforms, whether it be Snowflake, Databricks, Google BigQuery, or an established SAP environment. While this is a natural starting point, Khan cautions against recreating old patterns of vendor lock-in. He recommends prioritizing data that holds the most significance by focusing on preserving and injecting business context into operational and application data. Investing early in governance and semantics, including the definition of shared policies, access rules, and semantic models, is crucial before scaling pilot projects. Furthermore, businesses should champion openness and fabric-style interoperability over attempts to consolidate all data within a single stack. Khan advises against aiming for complete automation prematurely. "There is a new brave opportunity to really engage in the agentic and AI world," Khan remarks. "Fully automating [critical business processes] is maybe a stretch, because there’s going to be a lot of extra oversight necessary." Early successes are more likely to be found in less critical processes and with agents that operate on fresh, stateful data rather than outdated dashboards. As AI begins to deliver tangible value and adoption grows, leaders will face the strategic decision of reinvesting these gains to enhance top-line efficiency or to venture into new markets.