Last summer, a highly regarded study by MIT brought some sobriety and pragmatism to the previously euphoric AI industry. According to the study, at the time of the survey, only 5 percent of all projects with generative AI could demonstrate measurable business added value – even though the companies surveyed were very committed to AI and 90 percent of them had even seriously considered purchasing AI products.
This did not affect the companies’ commitment. AI transformation continues to be in full swing and AI pilot projects dominate the activity of IT and development departments. According to recent studies, the proportion of AI projects that meet or exceed their targeted ROI has increased to around 20 percent. But only about a third of all pilot projects make it into productive operation, Deloitte and McKinsey report in unison in corresponding studies. Only the AI pioneers, around a quarter of the companies surveyed by Deloitte, had managed to successfully integrate more than 40 percent of their AI projects into business operations by late summer 2025.
In order to tackle the next chapter of AI innovation with agentic AI, the existence of data and AI governance is an absolute prerequisite.
shutterstock/Gorodenkoff
Pilot projects in reality check
Various studies, as well as the experiences of many experts, show that the reasons for the failure of AI pilot projects are similar across all industries and company sizes. One of the most common mistakes is that AI projects are set up as IT department initiatives without consultation with the relevant specialist department. It goes without saying that this cannot go well.
“IT generally does not know the specific problems in specialist areas such as finance, production, sales or quality assurance,” explains Dan Hoffmann, Product Specialist for Data & AI at OEDIV, a company in the Oetker Group. “That’s why discussions to explore use cases must be held together with the specialist departments. Especially with the decision-makers who have financial responsibility for their area. Because they are the ones who can best estimate what the monetary impact of solving a current problem would be.”
Another mistake is to be too blinded by the capabilities of large language models (LLMs), such as those used in ChatGPT or Claude. “Although LLMs have enormous capabilities, they are not knowledge databases and do not have long-term memory,” explains Hoffmann. “For example, they can deliver plausible results that are not necessarily correct. False expectations of such models can lead to bad investments and disappointments.”
No ROI without a solid data foundation
Aside from such errors, Deloitte identifies a fundamental mismatch between the requirements of a pilot project and those of live operations as the main reason for AI project failure. Pilot projects usually run in an isolated environment and work with a cleaned, often static data quota. Live operation, on the other hand, has much higher requirements – from the necessary computing resources and the performance of the data infrastructure to integration with other systems to operational security, governance and compliance.
A pilot project that is too small can also lead to a misestimation of the actual operating costs in live operation. “Companies should not make the mistake of using the operating costs of a prototype as a metric for operating the live application,” warns Hoffmann. “Every API call, every human review step subsequently causes costs that may not have been incurred in the prototype.” You should be aware of this, because with this attitude, subsequent errors are inevitable.
“Many IT departments use powerful tools for their pilot project and use them to implement smaller use cases in the hope of impressing their management and getting the largest possible budget for building their AI infrastructure,” explains Dan Hoffmann. “The use case may work as a pilot project, but it usually fails when it is transferred to productive operation because the prerequisites are not met. Most often, there is a lack of an integrated database that can be completed across departmental and process boundaries. Without it, the AI is only of limited use because it cannot take the overall context of a task into account.”
Without governance, there is no control
If an AI-capable basic infrastructure does not exist, the targeted ROI also melts away very quickly. AI pioneers recognized this need early on. In the McKinsey study, 46 percent of companies that already have live AI applications said they have a centralized data infrastructure and governance, and another 39 percent have largely consolidated their resources. 57 percent of these companies have set up a central authority for the areas of risk management and compliance. The result: Companies that have adapted their infrastructure base to the requirements of AI applications benefit the most from their AI projects by increasing their productivity, automating routine activities, reducing operating costs, increasing customer satisfaction and increasing their revenue.
However, the use of so much different data in company-wide processes requires appropriate data governance. “The more AI intervenes in decisions, communication or automation, the more important the question of where data comes from, who is allowed to see it, how up-to-date it is and under what rules it is used becomes,” explains Hoffmann. “An integrated database is therefore not synonymous with an unregulated data lake, but with a structured, controllable and comprehensible information base. For companies, this is not just a compliance issue, but a prerequisite for safely integrating AI into critical processes.”
The task of data governance is to ensure data quality and consistency, monitor compliance with access authorizations and data protection standards such as the GDPR, take copyrights and licenses into account, avoid biases and ensure the scalability, maintainability and timeliness of the database. Data governance should be understood as part of overarching AI governance. The latter is in turn one of the most important means of carrying out the risk management required by law for many AI applications. For this purpose, the OECD, the EU and the US standards authority NIST have each issued guidelines, a legal framework and a risk management framework.
Set long-term goals
In order to tackle the next chapter of AI innovation with agentic AI, the existence of data and AI governance is an absolute prerequisite. Agentic governance builds on this foundation to meet the significantly higher requirements for data integration and interoperability. “An agent that not only responds but also executes tasks must reliably understand system states, rules, permissions, process contexts and current business data,” says Dan Hoffmann.
“Governance and trust are the foundation for growth,” writes the management consultancy Bain & Company in a current advisory on this topic. Permissions, access to tools and the decisions or transactions that AI agents can carry out would therefore have to be within defined parameters. “Governance also requires identity and authentication models that apply the least privilege principle not only to human users, but also extend it to autonomous agents.”
Speaking of innovation: The marketing of IT/AI providers meant that many managing directors and board members relied solely on short-term increases in the productivity of their employees and faster, more efficient processes. This expectation is justified and was confirmed by 80 percent of those surveyed in the McKinsey study. But companies that are currently deriving the most benefit from AI are more clearly recognizing the long-term role of AI for their company. In the study, they cited growth or innovation as additional goals of their AI commitment significantly more often than the rest of the respondents)
Check your AI foundation
