This is the trap that companies should avoid: No CIO wants to have to explain that the company has successfully put AI into operation – but no business value has been created because the vendor bill has eaten it up. Many companies have already experienced a similar situation with regard to the cloud – repeating the same mistake with AI is not a viable option.
Hybrid AI – the natural end goal
It is increasingly becoming clear that the future of enterprise AI lies neither exclusively in the public cloud nor on-premises. Rather, in many cases this future is hybrid: the market is maturing beyond ideological divides and evolving in a direction where workloads are placed based on economics, governance, latency and control.
This change is significant because not every AI problem requires a huge hosted model – on the contrary: more and more users are finding that smaller, domain-specific AI models perform just as well and often even better at specific business tasks. Some use optimized models, others rely on classic machine learning and predictive systems. Still others combine retrieval techniques with Small Language Models (SMLs). And still others develop limited models that are strictly tailored to specific operational domains. These systems are often much better suited for a private infrastructure: They are closer to the company data, can be optimized for predictable workloads and do not come with an open, token-based billing model.
This is especially true if the AI model is used frequently within internal business processes and not just occasionally by a limited number of users. In other words, companies aren’t just choosing private AI because they don’t like public cloud pricing. They choose this because they learn to develop AI systems that meet the needs of the business, rather than defaulting to what is easiest to use (from the outside).
Security and governance as additional drivers
The cost of AI may be the most pressing concern for most users – but not the only one. Security and governance are becoming equally important drivers as user companies become increasingly uncomfortable with the idea of sensitive information being processed through publicly available AI tools, APIs and user workflows that are difficult to monitor and control.
And this concern is not abstract: Confidential information regularly finds its way into public AI interfaces because employees strive to increase their productivity. For example, dev teams sometimes work faster than the guidelines can keep up, or business areas initially introduce tools outside of governance. This increases the risk of data leaks, compliance violations and security incidents that are directly related to the use of AI.
