Red Hat and Samsung drive enterprise AI infrastructure strategy – News

Enterprises looking to push AI infrastructure from lab experiments to production-ready solutions face a familiar bottleneck: Data access, system compatibility and performance at scale.

Red Hat Inc. is tackling this challenge head-on through collaborative efforts with hardware and chip partners — most recently, Samsung Electronics Co. Ltd. — to optimize artificial intelligence and memory technologies in real-world enterprise environments.

“I think it all starts with large language models,” said Stephen Watt (pictured), vice president and distinguished engineer, Office of the CTO, at Red Hat. “I think we had this sort of era of predictive AI, and now with generative AI, I think there’s a whole lot of … new applications … and … interesting new use cases in three different areas: training, fine-tuning and inference. Last year, we announced the InstructLab, which was democratizing fine-tuning models. With our Neural Magic acquisition, we’ve got a lot more into inference, and that’s about serving models and creating value for applications in the enterprise.”

Watt spoke with theCUBE Research’s Rob Strechay and theCUBE’s host Rebecca Knight at Red Hat Summit, during an exclusive broadcast on theCUBE, News Media’s livestreaming studio. They discussed Red Hat’s evolving AI infrastructure strategy, open-source innovation and its partnership with Samsung. (* Disclosure below.)

Scaling AI infrastructure with context and control

Red Hat’s partnership with Samsung centers on enabling compatibility between reference architectures to support the emerging Compute Express Link memory market. The goal is to create an ecosystem where enterprises can confidently deploy AI workloads across hybrid infrastructure, especially where performance and latency requirements make data locality critical, according to Watt.

“I would say it’s all about context,” he said. “There’s retrieval, augmentation, [retrieval-augmented generation], and then [Kafka] Raft, which is applying RAG with fine-tuning. We’ve got an emerging story around that with the upstream Llama Stack project, where we’ve just done a lot of work upstream to enable all of that.”

As AI scales across edge, data center and cloud environments, Red Hat is leaning into its distributed systems pedigree to tame inference sprawl. The company is prioritizing engineering strategies that make institutional knowledge more accessible to AI models and exploring new architectural patterns for seamless model integration, according to Watt.

“I think there’s two specific areas going back into context again, he said. “One is vector databases. You take those [extract, transform, load] pipelines and you chunk all your documents back into those vector databases. Once you do that, you’re able to basically take what you institutionally know within your organization [and] add it into something that’s accessible from the large language model. The second thing that’s really interesting is the evolution of service-oriented architectures … to hook those into the large language model — I think those two things together are really exciting.”

Open-source leadership remains foundational to Red Hat’s approach to AI infrastructure.  As innovation accelerates across model development and AI deployment, the company continues to invest in upstream communities that support transparency, trust and long-term viability for applied AI solutions, according to Watt.

“The rate of innovation, new projects, the creative destruction that’s happening … our role is to basically create a steady pipeline that businesses can use to consume software where it’s stabilized and safe,” he said.

Here’s the complete video interview, part of News’s and theCUBE’s coverage of Red Hat Summit:

(* Disclosure: TheCUBE is a paid media partner for Red Hat Summit. Neither Red Hat Inc., the primary sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or News.)

Photo: News