AMD Ryzen AI NPUs are now running LLMs on Linux with the recently debuted Lemonade 10.0 server and FastFlowLM 0.9.35 adding Linux support. In addition to those software components, AMD engineers have also been developing MLIR-AIE as a compiler toolchain for AMD AI Engine devices such as Ryzen AI NPUs in leveraging LLVM-based code generation with the Multi-Level Intermediate Representation (MLIR). Out today is MLIR-AIE v1.3 with some notable new features.
MLIR-AIE provides Python APIs and more for leveraging AMD NPUs as an alternative to the traditional Ryzen AI software workflows focused on AI inferencing. The hope has been with MLIR-AIE to also open up the Ryzen AI NPUs to digital signal processing and other non-AI/ML workloads thanks to the versatility of LLVM and MLIR. MLIR-AIE also ties into AMD’s Peano code. Beyond Ryzen AI NPUs, MLIR-AIE also benefits the other AMD-Xilinx accelerator products.
Thanks to the broad support for MLIR in the software ecosystem and even across hardware vendors, going the MLIR route allows versatility too for targeting other AMD wares such as Radeon/Instinct products via ROCm, CPUs via LLVM, etc.
New with MLIR-AIE 1.3 is a new C++ aiecc compiler to serve as an alternative to the project’s existing Python-based tooling. Code comments in the new AIECC C++ file sum up this new compiler as:
“This is the main entry point for the AIE compiler driver (aiecc). It orchestrates the compilation flow for AIE devices.
This C++ implementation provides similar functionality to the Python aiecc.py tool with the following architecture:
1. Command-line argument parsing using LLVM CommandLine library
2. MLIR module loading and parsing
3. MLIR transformation pipeline execution
4. Core compilation (xchesscc/peano)
5. NPU instruction generation
6. CDO/PDI/xclbin generation
7. Multi-device support”
The C++ aiecc compiler delivers better performance, supports C++17, can handle MLIR module loading and parsing, NPU instruction generation, ELF instruction generation, and other features. This README file has more details on the new AIECC C++ compiler.
MLIR-AIE 1.3 also has a number of AIE2 vector improvements around BF16, FMA, reductions, and tanh. There is also more robust runtime support, more IR features, and other improvements. Plus this release brings early native Windows support as an alternative to Linux use.
Downloads and more details on today’s AMD MLIR-AIE 1.3 release via GitHub.
