You can’t FinOps your way out of AI cloud costs – News

Scan any industry publication and the same story shows up: Cloud costs are out of control, and enterprises are scrambling.

The technology everyone bet on to drive growth is making the problem worse: Some 55% of respondents to a recent PricewaterhouseCoopers International Ltd. survey say they have yet to see any benefit from artificial intelligence tools.

The fix that’s often proposed is FinOps with its focus on better dashboards, tighter governance and smarter forecasting. But the waste keeps growing. Enterprises still burn more than a quarter of their cloud budgets, and though the tools measure the bleeding, they don’t stop it.

What’s missing is an honest look at what’s actually running up the bill. I spent two decades at Microsoft Corp. and SAP SE watching companies optimize the visible parts of the stack while the engine underneath quietly strained. Cloud costs today are the same story.

Cloud bills don’t spike in a vacuum. They map to the costs of data processing. AI has turned data processing into something cloud architectures were never built to handle.

Why AI breaks the cloud model

Before AI consumed every boardroom, enterprises spent a decade building cloud analytics. Datasets were structured, and workloads ran in batches on predictable schedules. Data processing was manageable because the economics worked.

AI blew that up. Batch became continuous, sample data became complete datasets and scheduled jobs became real-time pipelines over multimodal data. The volume, frequency and complexity of data processing have changed, but the underlying architecture hasn’t.

Here’s the part nobody talks about: Even after spending more every year, most enterprises process only a small fraction of their data in the cloud because running everything there would blow budgets wide open. They’re paying more for cloud and running it harder, but most of the data that’s actually needed for AI remains untouched.

The fix FinOps can’t reach

Companies bending the cost curve aren’t doing it with FinOps; they’re fixing the data processing layer.

Today’s engines were built for identical central processing unit clusters, but modern infrastructure spans CPUs, graphics processing units, field-programmable gate arrays and custom accelerators scattered across clouds. The software hasn’t caught up. Workloads still run on one-size-fits-all setups that can’t route jobs to the right hardware, leaving expensive accelerators sitting idle while CPU clusters max out.

GPUs rip through certain operations 10 to 100 times faster than CPUs, but only if the software knows where to send the work. When an enterprise’s data processing engine assumes CPU homogeneity in a heterogeneous world, they pay for next-generation hardware just to get legacy performance.

The solution is to fix that mismatch. Rebuild the foundation for what AI actually demands so that workloads are routed to the hardware that makes sense. The result is that costs drop hard. I’ve seen a major e-commerce platform that processes half a petabyte of data daily cut costs by 80% with no code changes and no migration. A social platform serving 350 million users cut costs by 50% using the same pattern.

What actually works

FinOps has a role, but dashboards, governance and forecasting are tools for tuning a working model, not fixing a broken one.

As long as AI pipelines run on infrastructure designed for batch analytics, costs will climb no matter how tight the governance is. You can forecast it, dashboard it and assign cost centers and chargeback teams, but the engine underneath is still wasting cash.

Enterprises that solve the data economics problem can process complete datasets at costs that don’t require quarterly budget battles. The rest will keep watching costs rise while returns shrink, staring at FinOps dashboards that show exactly where the money went but don’t tell them how to hold on to it.

Image: kalhh/Pixabay

JG Chirapurath is president of DataPelago Inc. and a former vice president in Microsoft Corp.’s Azure cloud. He wrote this article for News.