File system upgrades for AI-ready performance – News

Everyone in enterprise technology is familiar with the parallel file system; the question is how to modernize them for the needs of artificial intelligence and advanced analytics.

The collaboration between Los Alamos National Laboratory, PEAK:AIO Ltd. and Solidigm, a trademark of SK Hynix NAND Products Solutions Corp., is a sign of what’s to come for scalable storage systems. The goal, according to Mark Klarzynski (pictured, left), co-founder and chief strategy officer of PEAK:AIO, is to build a modern framework from legacy architecture.

“The reality is Solidigm makes amazing drives,” he said. “At the top end, network people make amazing networks. It’s not as magic as it used to be. Our job has been to recreate that traditional storage stack, that really just takes advantage of all the amazing work that Solidigm has done and get it out to the user in a way that they want to use it.”

Klarzynski, Gary Grider (right), High-Performance Computing Division leader at Los Alamos National Laboratory, and Alan Bumgarner (center), director of strategic planning for the Data Center Group and AI technologist at Solidigm, spoke with theCUBE’s Savannah Peterson at SC25, during an exclusive broadcast on theCUBE, News Media’s livestreaming studio. They discussed the evolution of the parallel file system and storage for AI. (* Disclosure below.)

Upgrading the parallel file system

Los Alamos National Laboratory manages the nuclear stockpile for the U.S., which means its people want to stay on the cutting edge of innovation. Grider describes them as pseudo venture capitalists who anticipate their storage needs by investing in, and working alongside, tech companies.

“We’re not necessarily in the build computers business,” he said. “We’re in the buy computers business. We tend to run into problems a few years before others do because of scale or needs. The only way we can field systems that are truly usable for us is to work with the industry to sort of pull the things we need along sooner than they might occur otherwise.”

Grider has helped PEAK:AIO and Solidigm to scale up, according to Klarzynski and Bumgarner. Those partnerships recently bore fruit in the form of an open version of Parallel NFS, a parallel network file system. The product is part of PEAK:AIO’s commitment to open source, according to Klarzynski.

“We need a newer standard that takes advantage of what people have today,” he added. “One of the biggest lessons I’ve learned with [Grider] is that it can’t be proprietary. The moment you make that proprietary, then it’s not a standard.”

As storage evolves to meet the needs of AI, Bumgarner foresees the next big bottleneck as networking. Now, that the industry is getting more bandwidth out of a single solid state drive, it has begun to scale out with hundreds of racks of SSDs, he explained.

“The further out you go, the more dense the memory is and the slower it is,” Bumgarner said. “As you get closer to the GPU, it gets smaller and faster. This whole thing needs to start lifting and making that happen and making it work in parallel. There’s many, many, many more years of innovation that we have in front of us.”

Here’s the complete video interview, part of News’s and theCUBE’s coverage of SC25:

(* Disclosure: Solidigm sponsored this segment of theCUBE. Neither Solidigm nor other sponsors have editorial control over content on theCUBE or News.)

Photo: News