Over the past two years AMD has developed the AMDXDNA accelerator driver in the mainline Linux kernel for supporting the AMD Ryzen AI NPUs. But when it comes to user-space software on Linux actually able to leave the Ryzen AI NPUs it’s been… extremely limited with nothing really useful besides some niche bits of code. Even AMD’s own software like their GAIA on Linux has used Vulkan with their iGPUs rather than any NPU support. But finally today there is a significant shift with the Ryzen AI NPUs becoming useful on Linux and able to handle LLMs.
The open-source Lemonade server for running LLMs today saw the release of Lemonade 10.0 and with it comes LInux NPU support for large language models as well as Whisper. Plus there is also native integration with Claude Code in Lemonade 10.0.
Lemonade for its Linux Ryzen AI NPU support is building off FastFlowLM as a means to “unlock Ryzen AI NPUs” as an NPU-first runtime built exclusively for Ryzen AI. FlastFlowLM with current-gen Ryzen AI NPUs can support context lengths up to 256k tokens. FastFlowLM 0.9.35 released this morning and it comes with official native Linux support.
Besides the Lemonade 10.0 server and latest FastFlowLM runtime, you also need to be using the Linux 7.0 kernel or the AMDXDNA driver back-ports coming to existing stable kernel versions due to some last minute accelerator driver tweaks. This Ryzen AI NPU support on Linux should work with all current AMD Ryzen AI 300/400 series SoCs.
There is this Lemonade documentation guide that outlines running LLMs on Linux with FastFlowLM and Lemonade.
I’ll be giving this AMD Ryzen AI NPU support on Linux a try in the days ahead with the hardware I have available. (Sadly the HP ZBook Ultra G1a review sample needed to be returned long ago for the Ryzen AI Max+ 395 in laptop form, but there is always the Framework Desktop and then the Ryzen AI 300 Strix Point hardware I have in the lab.) Hopefully all goes well with this long-awaited Ryzen AI NPU Linux LLM support for some benchmarking.
The timing of this is important with the Ryzen AI Embedded P100 series coming to market as well as the Ryzen AI PRO 400 series that given their markets will surely be seeing more Linux use than the typical consumer Windows deployments. Now there is finally a story to tell there around the Ryzen AI NPU LLM support on Linux.
