A fresh take is being pursued around allowing the Linux kernel’s out-of-memory (OOM) behavior to be customized using BPF programs.
Google engineer Roman Gushchin has sent out a “request for comments” patch series with a new take on allowing for OOM customization using BPF. There have been previous attempts at BPF-based OOM customization for adapting the kernel’s behavior but no solution yet ultimately deemed the best and suitable for upstreaming.
Gushchin explains in the RFC patch series:
“This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal, which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy showed that there are multiple opinions on what the best policy is. As it’s highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It’s a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years
several PSI-based userspace solutions were developed (e.g. OOMd or systemd-OOMd). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.”
We will see where this newest BPF-based OOM adaptation heads and if it’s ultimately accepted into the mainline Linux kernel.
