In the past we have seen Llama.cpp with Vulkan outperforming AMD’s ROCm compute stack in some of the large language model (LLM) AI benchmarks. Curious if anything has changed given the recent ROCm 7.1 release, I ran some benchmarks of an up-to-date Llama.cpp using the AMD ROCm back-end compared to the Vulkan back-end with the latest RADV driver. For this round of testing the Radeon AI PRO R9700 graphics card was used.
Curious how the ROCm 7.1 versus Vulkan performance is looking with Llama.cpp when using the latest ROCm and RADV drivers, I ran some fresh benchmarks the past few days for seeing if the ROCm performance is anymore decisive or how well RADV/Vulkan is looking for AI these days using the Mesa Git driver stack.
The AMD Radeon AI PRO R9700 running with the Threadripper PRO 9995WX was used for these straight-forward benchmarks of comparing the two back-ends on the same hardware.
