Cloud-native AI: Kubernetes powers next wave News

A case could easily be made that Kubernetes, the open-source container orchestration tool first introduced by Google LLC in 2014, was made for artificial intelligence. Support for large, complex environments? Check. Scalability? Kubernetes can automatically scale. Portability? The open-source tool can run in hybrid configurations of public and private instances. Support for traditional applications? Kubernetes helps deliver containerized and cloud-native apps. DevOps enablement? Check again.

The Google engineers who designed Kubernetes likely could not have foreseen the explosion of AI use cases 10 years later, yet the Cloud Native Computing Foundation’s most widely deployed project has emerged as a key resource for implementing artificial intelligence in the enterprise. Kubernetes serves as the central component in cloud-native AI — the process of building, deploying and managing AI and machine learning workloads utilizing cloud-native principles.

“Cloud native and AI are the most critical technology trends today,” said former Cloud Native Computing Foundation Executive Director Priyanka Sharma, when Kubernetes marked its tenth anniversary last year. “Cloud-native is the only ecosystem that can keep up with AI innovation.”

More than a decade after its debut, Kubernetes has evolved from a developer tool into the backbone of modern AI infrastructure. Its impact now extends across data centers, clouds and edge environments, reshaping how enterprises build and scale intelligent systems. Ahead of this year’s KubeCon + CloudNativeCon NA event, News takes a look at how Kubernetes’ continued evolution is powering the next era of open-source, AI-driven computing.

This feature is part of News Media’s ongoing exploration into the evolution of cloud-native computing, open-source innovation and the future of Kubernetes. Be sure to watch theCUBE’s analyst-led coverage of KubeCon + CloudNativeCon NA from November 11-13. (* Disclosure below.)

Red Hat leverages cloud-native resilience

The evolution of Kubernetes from niche tool to mainstream technology has been fueled by a need among organizations for performance that can balance cost, sustainability and security. These have become key factors as companies pursue experimental and enterprise-grade AI deployments.

“To support our customers, we have to make sure that they are able to run in a cloud-native and distributed way, data stacks and enterprise grade as well,” said Vincent Caldeira, chief technology officer for APAC at Red Hat Inc., during an interview with theCUBE. “There is this concept of resilience that is extremely important for people to manage. Kubernetes is very good at this, at orchestration, resilience, recovery.”

Companies such as the cloud platform provider CoreWeave Inc. have built their entire stack around Kubernetes. This has allowed them to leverage open-source cloud-native tools and create adaptable infrastructure that can grow rapidly with AI demand.

“We’re on Kubernetes and bare metal, and it helps us both scale,” according to Peter Salanki, chief technology officer of CoreWeave, during a conversation with theCUBE. “When customers come to us, it’s a familiar interface. They don’t have to learn proprietary APIs, and that really lets people hit the ground running and has allowed a scale we simply couldn’t do without it.”

Yet, despite the merits of Kubernetes, DevOps practitioners readily acknowledge that there is still work to be done. While containers have played a key role in application modernization strategies, there is a need for greater monitoring and observability, as seen in the rise of OpenTelemetry, now the second highest velocity project in CNCF after Kubernetes. This has been further validated in theCUBE Research’s survey of over 650 cloud-native application teams.

“In our AppDev research this year, we observed that 61% of organizations say their containerization strategy is now central to their application-modernization roadmaps,” noted Paul Nashawaty, principal analyst with theCUBE Research. “Yet only 27% report that their Kubernetes platforms are running with full production governance, security and observability baked in. What we’re seeing in practice is a two-speed reality: The infrastructure team is embracing containers rapidly but the operational maturity of that strategy, governance, metrics and threat detection is lagging far behind. At the forthcoming KubeCon + CloudNativeCon NA 2025, the spotlight will shift not just to deploying pods and services, but on how observability, DevSecOps and container runtime security must converge to make those deployments resilient, efficient and operator-ready.”

Lightspeed adds AI capabilities for developers

Kubernetes took an additional step forward on the operational side when Red Hat introduced a number of enhancements to its OpenShift platform. This included a new suite of context-aware, generative AI tools which can integrate assistance into developer workflows.

Red Hat Developer Lightspeed adds AI capabilities to Developer Hub and provides a toolkit for application migration. Lightspeed is integrated directly into the Red Hat OpenShift 4.20 web console and its conversational interface can help developers troubleshoot and investigate cluster resources.

“Lightspeed is simply a built-in AI assistant for OpenShift,” said Marc Curry, consulting (distinguished) product manager for the Cloud Platform Business Unit at Red Hat, in comments made during an October broadcast of the company’s “In the Clouds” program. “’Can you add this new thing, or can you configure it a certain way to connect to another cluster?’ That’s where it’s going.”

Another significant part of OpenShift 4.2 is the integration of Border Gateway Protocol. This networking protocol allows Red Hat customers to import provider network routes into OpenShift Virtualization and provide additional features to virtual machines.

“The biggest networking feature that we released in the 4.20 timeframe is BGP full support,” Curry said. “This is our second major feature to come out of something that we’re referring to as universal connectivity. It’s that effort that is focused on those really big major networking requests.”

Red Hat’s attention to OpenShift and networking makes sense, given where it is focusing on AI. In mid-October, the company announced AI 3, designed to manage AI workloads across data centers, clouds and edge environments. This hybrid cloud-native AI platform incorporates the llm-d open-source project that enables intelligent workload distribution for large language models. The goal is to leverage Kubernetes’ high-performance distributed inference framework and generate more efficient and scalable LLMs.

“We have a certain level of stability built into Kubernetes today, but the use case of generative AI is causing changes in a lot of the underlying components,” said Stu Miniman, senior director of market insights at Red Hat, during the “In the Clouds” broadcast. “Llm-d is helping to pull those things together.”

GKE drives scale and flexibility

In 2015, Red Hat adopted Kubernetes into OpenShift 3 for container orchestration. That same year, Google Kubernetes Engine was launched to simplify the process of deploying, managing and scaling containerized applications in Google Cloud.

Over the past decade, GKE has become a cornerstone of Google’s overall cloud strategy. GKE’s ability to handle complex AI inferencing workloads has allowed enterprise teams to iterate quickly and serve models at scale. Upgrades to support 65,000 node clusters and integration with platforms such as Cloud Run have paved the way for application building in the AI era.

“If you think about what GKE is, it’s a very complicated, very organized kitchen that has all the equipment you need,” said Eddie Villalba, outbound product manager at Google Cloud, during a recent interview with theCUBE. “But when I need to create that Beef Wellington, I can. When I need to create just a bunch of salad, I can. When I need to just serve web services, it’s easy; GKE was already built for that.”

The flexibility described by Villalba highlights the evolution of GKE into a driver for multicloud and hybrid strategies. As enterprises modernize infrastructure to meet accelerating AI demand, multicloud and hybrid are becoming the preferred playbook, according to theCUBE Research’s Nashawaty.

“I think the big area of growth for Google right now is we’re seeing it in our own research that 94% of organizations are using two or more clouds [and] 65% are using four or more clouds,” he said. “They have to adopt the multicloud hybrid cloud approach. GKE is a way to do this, to harmonize this across the platforms [and] make sure that it’s working.”

At the customer level, GKE’s balance of speed, scalability and savings has allowed firms such as HubX to leverage the technology for powering the mobile app market. The maker of popular apps such as DaVinci, Momo and PhotoApp, relies on GKE and Google Cloud to provide high performance for users and enable developer productivity.

“If you make a user wait 30 seconds from prompt to output, they won’t wait,” said Cem Ortabaş, co-founder of HubX. “To reduce churn, everything needs to happen in under 10 seconds. That’s the bar we’re measured against — and it directly impacts the bottom line. With GKE, we don’t have to worry about the underlying infrastructure. This frees up our developers to build and deploy applications at scale.”

Startups foster cloud-native innovation

While Red Hat and Google have taken prominent roles in advancing cloud-native innovation and infrastructure modernization, there are other firms making advancements in the market. Companies making notable moves in this area include Vultr, Elastic and Vast Data, along with Honeycomb, Backblaze, Spacelift and Union.ai.

Honeycomb’s observability platform provides engineers with code-level analysis from a heat map to improve visualization of key data points. In September, the company launched an AI-powered suite of products to help developers detect performance anomalies more rapidly.

Backblaze provides cloud storage and services built to accommodate developers. It recently announced a set of enterprise web console and role-based access controls for cloud-native security and simplified management.

Spacelift, an infrastructure-as-code orchestration platform, is enabling codeless infrastructure provisioning for cloud workloads. The company released enhancements in October that provided an agentic infrastructure deployment model for allocating cloud resources using natural language commands.

Union.ai is the creator of Flyte, an open-source AI workflow orchestrator. When Amazon debuted S3 Vectors in July as the first cloud object store with native support for querying vectors, Union noted that Flyte was already supporting it.

Innovation by startups, along with key moves by major industry players such as Red Hat and Google, underscore the increasing influence of Kubernetes and the cloud-native community in the AI revolution. This is about a lot more than speed. The basic infrastructure necessary to run and manage AI effectively will require organizations to close the gap between adoption and operational maturity.

As theCUBE Research has noted, organizations are facing a fundamental pivot in this cloud-native era.

“Their next release cadence isn’t just about ‘faster,’” Nashawaty said. “It’s about ‘secure from day zero,’ which means integrated security (DevSecOps), native container monitoring (observability), and automated compliance governance layered into every image, cluster and service.”

(* Disclosure: TheCUBE is a paid media partner for the KubeCon + CloudNativeCon NA event. Neither Red Hat and Google, the headline and premier sponsors of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or News.)

Image: Microsoft Designer/ News