A new set of Linux kernel patches posted today work to improve the nested VMX performance for benefiting Intel processors making use of KVM virtualization.
Amazon engineer Fred Griffoul posted a set of five patches aiming to address performance issues around nested VMX (Intel) when handling unmanaged guest memory. In particular, memory passed via the “mem=” parameter or guest_memfd for non-confidential computing virtual machines.
It was found that with the current Intel VMX code there is significant performance overhead due to expensive remapping operations. With nested VMX usage the system is accessing guest pages during L2 VM entry/exit cycles leading to much overhead.
As part of the fix is introducing a cache for L1 APIC pages for enhancing the performance of the unmanaged guest memory:
“This eliminates expensive memremap/memunmap cycles for each L2 VM entry/exit, providing substantial performance improvements when using unmanaged memory such as guest_memfd or memory passed with mem= kernel parameter.
The persistent caching approach maintains correctness through proper invalidation detection while avoiding the overhead of repeated mapping operations.”
No specific benchmark numbers were provided publicly for quantifying the performance impact of this overhead for nested VM use-cases.
See this patch series for those interested in the work to enhance the nested VMX performance under KVM for Intel CPUs on Linux.
