In addition to AMD being involved with Red Hat on the new llm-d open-source project for Gen AI, AMD and Red Hat also announced today further collaboration around open-source GPU/accelerator support for AI workloads.
AMD Instinct accelerators are now fully-enabled on Red Hat OpenShift AI. AMD Instinct MI300X GPUs can also be used with Red Hat Enterprise Linux AI while they are also working on further collaborating around the upstream vLLM community for further enhancing open-source AI inference.
Red Hat and AMD will be jointly working together on the following areas:
“Improved performance on AMD GPUs: By upstreaming the AMD kernel library and optimizing various components like the Triton kernel and FP8, Red Hat and AMD are advancing inference performance for both dense and quantized models, enabling faster and more efficient execution of vLLM on AMD Instinct MI300X accelerators.
Enhanced multi-GPU support: Improving collective communication and optimizing multi-GPU workloads opens the door to more scalable and energy-efficient AI deployments, which is particularly beneficial for workloads that require distributed computing across multiple GPUs, reducing bottlenecks and improving overall throughput.
Expanded vLLM ecosystem engagement: Cross-collaboration between Red Hat, AMD and other industry leaders like IBM helps accelerate upstream development to propel continuous improvements for both the vLLM project and AMD GPU optimization, further benefiting vLLM users that rely on AMD hardware for AI inference and training.”
AMD Instinct will also be supported by the Red Hat AI Inference Server.
More details for those interested in this latest AMD and Red Hat collaboration via today’s press release.
