Intel’s accelerator efforts in recent generations of Xeon processors have been challenging to say the least. From limited software support and configuration obstacles to some current-generation accelerators not being safe for VM use due to security issues to having to deal with higher latency and other nuances if wanting to achieve decent performance. One of the Linux kernel users of Intel’s QuickAssist Technology “QAT” accelerators has been the FSCRYPT code for native file encryption support. But the Linux kernel with the FSCRYPT usage is now demoting the Intel QAT accelerator support along with other problem-causing accelerator drivers.
Sent out on Sunday was the FSCRYPT pull request for Linux 6.17 with one change standing out:
“Drop the incomplete and problematic support for asynchronous algorithms. These drivers are bug-prone, and it turns out they are actually much slower than the CPU-based code as well.”
This caught me by surprise as I happened to miss seeing the related patches back when they were floated on the mailing list. The main one to look at is “fscrypt: don’t use problematic non-inline crypto accelerators”.
Google engineer Eric Biggers basically found the accelerators like QAT with Xeon Emerald Rapids to be a big headache for FSCRYPT file encryption and the performance to be much worse than the AVX-512 VAES optimized code path. Eric Biggers is responsible for a lot of AVX-512 optimizations and other optimizations for different CPU ISA extensions to the Linux kernel cryptography code. In the context of FSCRYPT, using QAT accelerators leads to horrible performance that even the plain C code can outpace but that the AVX-512 VAES path puts everything else to shame on Intel Xeon Emerald Rapids:
Biggers also found the Qualcomm Crypto Engine on the ARM side to perform poorly too. Biggers further added on that patch message:
“There are two major issues with these drivers: bugs and performance.
First, these drivers tend to be buggy. They’re fundamentally much more error-prone and harder to test than the CPU-based code, and they often don’t get tested before kernel releases. Released drivers have en/decrypted data incorrectly. These bugs cause real issues for fscrypt users who often didn’t even want to use these drivers.
…
Second, the CPU-based crypto tends to be faster, often *much* faster. This may seem counterintuitive, but benchmarks clearly show it. There’s a *lot* of overhead associated with going to a hardware driver, off the CPU, and back again.
…
So, using the “accelerators” is over 50 times slower than just using the CPU. Not only that, it’s even slower than the generic C code, which suggests that even on platforms whose CPUs lack AES instructions the performance benefit of any accelerator would be marginal at best.
…
As of this writing, the Crypto API prioritizes qat_aes_xts over xts-aes-vaes-avx512. Therefore, this commit greatly improves fscrypt
performance on Intel servers that have QAT and the QAT driver enabled. qat_aes_xts is going to be deprioritized in the Crypto API (like I did for xts-aes-qce recently too). But as this seems to be a common pattern with all the “accelerators”, fscrypt should just disable all of them.”
Biggers notes there that the QAT AES-XTS code is also going to be de-prioritized in the Linux Kernel Crypto API.
While this code is being submitted for Linux 6.17, the patches are marked as “Cc: stable” so they will also be back-ported to the current Linux kernel stable series in the coming days.
