It’s been a while since having any improvements to talk about for the MGLRU multi-gen LRU functionality for the Linux kernel to optimize page reclamation and help with system performance especially when enduring memory pressure. But this week a Tencent engineer posted some very promising patches for further enhancing this kernel feature.
Kairui Song of Tencent posted a set of eight patches for improving the reclaim loop and dirty folio handling within MGLRU. Here’s the exciting part:
“This series cleans up and slightly improves MGLRU’s reclaim loop and dirty flush logic. As a result, we can see an up to ~50% reduce of file faults and 30% increase in MongoDB throughput with YCSB and no swap involved, other common benchmarks have no regression, and LOC is reduced, with less unexpected OOM in our production environment.
Some of the problems were found in our production environment, and others are mostly exposed while stress testing the LFU-like design as proposed in the LSM/MM/BPF topic this year.”
A 30% improvement to MongoDB throughput is exciting along with reduced code complexity of MGLRU in the end. Those tests were done on NVMe storage and the patch cover letter went on to add when using slower devices like HDDs, they observed a greater than 100% gain for some workloads.
Quite nice work. Those interested in these patches can find them out for review on the Linux kernel mailing list.
