The jemalloc memory allocator “malloc” implementation has been popular for HPC and server use down to desktop use in apps like Firefox. Jemalloc has proven over the years to be effective on delivering better performance and scalability while enjoying lower memory usage and less fragmentation than alternative malloc implementations. Meta recently announced that they are renewing their investment into jemalloc.
An announcement made on the Meta Engineering Blog that is only making the rounds today is word of their renewed investment and commitment to jemalloc. Facebook/Meta has been using jemalloc in their infrastructure for nearly two decades. For over a decade they have invested in jemalloc’s development and maintenance. But in more recent times, there has been less to report on jemalloc’s progress.
But Meta announced this month they are now renewing their focus on jemalloc and working to modernize the codebase and reduce its maintenance burden. With the open-source community they are hoping to take jemalloc much further.
“High leverage comes with high stakes. On the spectrum of practical versus principled engineering practice, foundational software components like jemalloc need the highest rigor. With the leverage jemalloc provides however, it can be tempting to realize some short-term benefit. It requires strong self-discipline as an organization to resist that temptation and adhere to the core engineering principles.
In recent years, there has been a gradual shift away from the core engineering principles that have long guided jemalloc’s development. While some decisions delivered immediate benefits, the resulting technical debt eventually slowed progress.
We took the community’s feedback to heart. In the spirit of collaboration, we have reflected deeply on our stewardship and its impact on jemalloc’s long-term health. As we’ve met with some members of the community, including the project’s founder, Jason Evans, to share our introspection and how we are changing our approach. We’ve started an effort to remove technical debt and rebuild a long-term roadmap for jemalloc.”
Moving forward they are planning to reduce jemalloc’s technical debt, continue working on enhancing jemalloc’s hugepage allocator (HPA) mode for better using Linux’s Transparent Hugepages, increase memory efficiency, and also further AArch64 CPU optimizations.
Those wanting to learn more about this renewed investment can find out more via the Meta Engineering blog.
