An exciting new Linux kernel patch series was posted today for testing… Introducing support for cache-aware load-balancing. The patch comes from a veteran Intel Linux engineer but this cache aware load balancing may also prove very applicable for AMD Linux users for EPYC and Ryzen processors.
Intel engineer Peter Zijlstra posted the “request for comments” (RFC) patch series today for introducing cache-aware load balancing. As he explained, the code has been in the works for a while albeit only now making it to the mailing list. Peter wrote:
“One of the many things on the eternal todo list has been finishing the below hackery.
It is an attempt at modelling cache affinity — and while the patch really only targets LLC, it could very well be extended to also apply to clusters (L2). Specifically any case of multiple cache domains inside a node.
Anyway, I wrote this about a year ago, and I mentioned this at the recent OSPM conf where Gautham and Prateek expressed interest in playing with this code.
So here goes, very rough and largely unproven code ahead :-)
It applies to current tip/master, but I know it will fail the __percpu validation that sits in -next, although that shouldn’t be terribly hard to fix up.
As is, it only computes a CPU inside the LLC that has the highest recent runtime, this CPU is then used in the wake-up path to steer towards this LLC and in task_hot() to limit migrations away from it.
More elaborate things could be done, notably there is an XXX in there somewhere about finding the best LLC inside a NODE (interaction with NUMA_BALANCING).”
Gautham and Prateek noted in the patch cover letter are some of AMD’s Linux engineers. So even if Peter doesn’t have the bandwidth to carry this to the finish line in the near-term and get it mainlined, hopefully the AMD engineers will step up and test out these patches and get it tuned up into shape for upstreaming. AMD EPYC should especially benefit and to a lesser extent Ryzen (X3D) processors from this cache-aware load balancing and it will be interesting to test out this code once it hopefully evolves into good shape for seeing its performance benefits.
For now this cache-aware load balancing is sitting out on the Linux kernel mailing list for anyone wanting to test it.
