The LTTng tracing toolkit is twenty years old this year and it’s seen significant adoption by different hyperscalers and other notable organizations like IBM and Sony and Siemens beyond basic end-users and administrators for system tracing/debugging. While having many successes over the past two decades, the kernel modules remain outside of the kernel tree. Even with around four different upstreaming attempts to get the LTTng code into the mainline kernel, it still has not happened. A fifth attempt began today but still looks like it could be an uphill battle.
Mathieu Desnoyers who started LTTng brought up the idea again today around upstreaming the LTTng kernel tracer as its own subsystem within the mainline kernel. Past efforts to get LTTng upstream rather than being based on out-of-tree modules have largely failed due to it being around 75k lines of new code and thus a major undertaking to review. Some have also suggested LTTng be integrated into the Linux Perf subsystem or better unified with existing Linux kernel subsystems.
Desnoyers raised the idea and gauging the feedback for upstreaming the LTTng kernel modules into the mainline kernel via this RFC LKML thread. As for how to get it done, the preference would appear to be on bulk upstreaming all of the code to easily become a drop-in replacement to the lttng-modules currently in use and maintaining the existing user-space ABI. Meanwhile if trying to incrementally upstream the code, Desnoyers noted that it would be a large commitment and not necessarily funding to make it feasible as well as uncertainty if it would all work out and prove worthwhile. Desnoyers also raised the possibility of submitting the LTTng modules first to the kernel staging area while all of the code is cleaned up to mainline standards.
But Linus Torvalds already responded to the prospects of bulk upstreaming LTTng and isn’t exactly positive:
“Honestly, I don’t see the point.
The reason the current tracting infrastructure got merged was that people were willing to do it incrementally.
I was hoping that there would be some kind of eventual merging of the different ring buffers etc. That was discussed as a hopeful end goal originally, but here were are, decades later, and it never happened.
And honestly, I am NOT interested in just having two different tracing models.
If people need two tracing models, then the other one will be out of tree. It’s that simple.
Because if people haven’t been able to agree on common models in the decades past, I really don’t see the point in maintaining two models indefinitely side-by-side in the upstream kernel.
So as far as I’m concerned, this discussion is not a discussion. Either there’s a way to merge things incrementally with SHARED infrastructure, or there isn’t.
No “two different and disjoint trace buffers”.
No “two different and disjoint trace interfaces”.
And very clearly – based on history – that unification will never happen.”
Torvalds further commented after the idea was raised that it could be easier to merge tracing infrastructure in the kernel once the LTTng code was in-tree:
“That’s not a bet I’d take.
If people haven’t unified this in the last two decades, I’m not going to take the argument of “hey, merge it because *then* it will be unified”.
Because honestly, that sounds like a total fairy tale to me: “the princess came along and kissed the toad, and he turned into a beautiful [prince], and they lived happily ever after”.
So no. I don’t believe in fairy tales. Not when we have two decades of “that didn’t happen”.
If people can unify this and merge it incrementally, that’s one thing.
Until then, you’re just making stuff up.
“Show me the code”, in other words.”
So while LTTng has been around now for 20 years and has much user/corporate adoption, the likelihood of its kernel modules reaching the mainline kernel at least in the near-term appear to be still very slim given the initial comments by Linus Torvalds.
Those wanting to learn more about the current LTTng feature set can do so via LTTng.org.
