The Java Ecosystem Remains Ever-Green By Continuously Adapting to Developers’ Needs

Transcript

Olimpiu Pop: Hello, everybody. I’m Olimpiu Pop, an InfoQ editor, and I have in front of me, Kevin Dubois and Thomas Vitale, to discuss the shortest path these days in the cloud-native ecosystem from code to production. So, guys, let’s take it alphabetically and introduce ourselves. Kevin?

Kevin Dubois: Hello, I’m Kevin Dubois. I’m a developer advocate at Red Hat for the moment, at least. Our Java team is moving to IBM. So, by July, I’ll be mentioning that I’m from IBM, even though we’ll still be representing Red Hat as well. However, my background is in software development, and my passion has always been developer experience, specifically how we can make a smooth transition from local development to production. That’s what we’re covering today, too.

Thomas Vitale: Yes. Hi, everyone. I’m Thomas Vitale. I am a software engineer. I work at a software company called Systematic in Denmark. I am passionate about anything cloud-native and Java. I got sidetracked with all this interest of mine a while ago, writing a book about Cloud Native Spring, and now I’m writing a book about developer experience with my friend Mauricio Salatino. I am particularly passionate about the path to production and ensuring that application developers have the best possible experience when working with cloud-native technologies, such as Kubernetes.

Java, the de facto enterprise language, evolves to meet developers’ needs [01:49]

Olimpiu Pop: Great, thank you for this, guys. I was watching your presentation at KubeCon, and it was nice to see all those things; that took me back to a couple of conversations. One of them was the one I had with Max, whom I won’t pronounce by his middle name, Andersen, the team lead for Quarkus, because the last time I pronounced his name, it went horizontal. I didn’t pronounce it correctly, so I’ll skip that. The other one is at the point where the biggest contender, and probably the only contender, in the serverless space is Node.js. And I know that Quarkus originated from the question: What can we learn from other ecosystems to make Java better? So, Kevin, how do you feel about it? Are we closer to regaining some momentum for the Java language in the serverless space?

Kevin Dubois: I mean, I think so. I’m writing a book on the topic of serverless Java, so, of course, there’s some merit to it, but I think we’re seeing a lot of advancement in the entire Java ecosystem, right? That’s the nice thing about Java: it keeps evolving with time, and it is the de facto standard for enterprise development. So, if there’s something that enterprise users can use, then Java tries to answer that call. Now, regarding Quarkus specifically, that’s one of the reasons we started this project. We saw this little gap in the Java space in terms of a lot of people were using the Spring framework, and there’s a lot of great stuff about Spring, but one thing it wasn’t very good at was the startup speed and the memory usage because it wasn’t designed for that, right?

And so, with Quarkus, the main goal, at least at the start, was to figure out how to make Java startup faster and use less memory to utilize these cloud-native deployment targets. Initially, it was Kubernetes, but also for serverless, because with serverless, we want something that starts up quickly, as otherwise we can’t scale in the way we want to use serverless. We want something that, if there’s a lot of demand, we have a lot of instances that can handle that burst of traffic. And then, if the demand stops, we scale down, allowing us to use these systems flexibly.

And the billing system of serverless is also based on resource usage. So, we wanted something that’s much more compact. That was a pretty good success with Quarkus, as it moved a lot of the tasks that used to happen during startup time to the build time. And then that had the added benefit of making it also really easy to perform native compilation of these Java applications with GraalVM, which allowed for significant performance improvements. And then we see that, actually, now that these natively compiled Quarkus applications typically start up faster than Node.js. So, we’ve come a long way.

Outside of Quarkus. There are also several advancements in terms of startup time and memory. So, we see that there’s a lot of good stuff coming from the Java world to keep up with the demand.

Olimpiu Pop: Great. Thomas, you’re not actively working, at least from what I know about Spring, but you did mention that you wrote a book about Spring. So, what’s your feeling? How is this gap being filled? Because if you look at that, we can say that Kevin is now the competition, the Quarkus side of things. So, how do you see the other angle?

Thomas Vitale: In general, I am pleased to see the advancements in the Java ecosystem because, as Kevin said, whenever innovations or new technologies emerge around Java, Java continues to evolve and advance to support new use cases. I’m actively contributing to the Spring AI project, which integrates AI capabilities into Java applications. It’s great to see the vast Java ecosystem, including Semantic Kernel and LangChain4j, offering extensive support from both frameworks and the Java platform itself. Like the last few versions of Java, which have introduced many useful features that really bring Java to a whole new level when it comes to AI use cases.

From that perspective, being a Java developer is exciting. Regarding serverless in general, and specifically the Spring Boot side, there is support for GraalVM, which enables native compilation, which is awesome. You get instant startup time, reduced memory consumption, and reduced surface attack. That’s really important—that security aspect. But on top of that, the Java platform itself is working towards making startup time faster and reducing memory consumption.

So, overall, there’s a lot of innovation happening in the Java world, so I am excited to be a Java developer at this moment in time. I feel like I’m not missing out on much.

Production deployments are not a big event anymore, as the focus is on quick feedback loops [07:10]

Olimpiu Pop: Yes, that’s my feeling overall as well. And during KubeCon, one of the points that everybody was discussing was green, specifically having the ability to start faster, shut down gracefully, and then have a snapshot from which to start, allowing you to unplug some of the machines. We can explore ephemeral environments more thoroughly these days and ensure that our code is, well, making everybody happy.

The green side of the equation involves people putting money into the operational side, and last but not least, we have been speaking a lot lately about developer experience. And obviously, one of the points that is important for everybody is a shorter path to production. You both mentioned that during your introductions, those points were important. What do you think are the most important aspects when building something and achieving a fast feedback loop? How would you look at that, Kevin?

Kevin Dubois: I think it’s super important. Thinking back, I’ve been developing professionally for, oh, I think about 20 years. So, a while ago, and I remember when I started going to production, it was this big event because we knew stuff was going to go wrong. There were a lot of manual things that we needed to do, and the environment that we were developing on locally was utterly different from our production environment. So, even though we tried to test stuff on our local machine, we would have all these nasty surprises go into production. And now, fast-forward 20 years ago, yes, some stuff still happens, but I feel like we have a lot more control over that whole cycle and being able to reproduce a lot of the stuff that’s going to happen in production on our local machine, or at least in the … let’s say the inner loop or the close outer loop where we can also automate a lot of stuff with CI/CD and GitOps and stuff like that.

But I think in terms of local development, the advent of containers has been super important. I mean, I think when I started working with containers about 10 years ago with Docker, I containerized our environment that we had, kind of these different services running in our testing cloud environment. Yes, we already had EC2s and other resources, but it was still very clunky. And so when I containerized all these different components, we were able to run those on our local machine, reproduce all these different test cases that we were never able to do, and then build that into pipelines.

The amount of time that we spent debugging on production has gone down quite a bit thanks to that. And then also thanks to all these observability stacks that we have now that we can also reproduce, it makes it super nice. And then of course, adding the developer experience of stacks like Quarkus that allow you to also deploy to these environments much more easily is the cherry on top, I think.

Olimpiu Pop: How about you, Thomas? What do you have in your developer toolbox?

Thomas Vitale: Yes, totally agree with Kevin about containers. They have been a real game changer. On top of that, I think the introduction of this project called Testcontainers improve things even further. Testcontainers is a polyglot framework, so it’s not just Java-specific, even if it started in the Java world, but it gives you the possibility to programmatically provision these containers as part of your application. So, you have the same application lifecycle when you start your application. You don’t have to worry about starting up separate containers maybe for a PostgreSQL database, or maybe you want observability, you want to spin up Grafana, or maybe you’re working with an AI application and you want to run an Ollama instance to serve some models. Testcontainers integrated with frameworks like Quarkus or Spring Boot basically gives a very nice developer experience because you don’t even have to think about it.

You start your application and immediately all these services are provisioned for you. So, you can immediately go into the development flow where you are implementing the business logic, maybe a new feature, bug-fixing and all those services are part of the lifecycle.

Then you shut down the application and all those services are also shut down automatically. That’s great, both at development time but also testing time. I remember having to test integrations with databases, using some in-memory test-only databases or with some mocks. And then you go to production with the real database and you find out some incompatibilities because, of course, it’s a different system, but now you can actually use the same exact database across the entire software lifecycle. And that’s really, really important. It’s not only a matter of producing higher-quality software, but also getting a better experience as a developer. So, I enjoy my work more because I don’t have to spend too much time on all these extra activities that distract me from the core of my task, but also that frustrate me.

I think a core property of a good developer experience setup is you don’t get frustrated when you use it. How many times has it happen that we tried out a tool and we got frustrated because it was impeding, it was creating friction in our workflow? I think today we have lots of available tools and strategies in order to prevent it. Unfortunately, it’s not always prioritized or maybe some tools are not always known. And that’s also why it’s really important to talk about this topic, and I really enjoyed talking about it with Kevin at KubeCon.

Olimpiu Pop: Great. So, there is no more H2 SQL to be used, so we can just use Testcontainers and then we are getting a better experience in close to what the production environment would look like probably, right?

Kevin Dubois: Yes-

Thomas Vitale: Yes, exactly.

Kevin Dubois: What’s also nice is that as a new developer coming onto a project that was, in the past, also a big pain to figure out how I need to set up everything and how everything is configured? And these days the pain is also … I mean, I wouldn’t say it’s completely gone, but it’s also quite a bit less. I remember in the past it was sometimes weeks to set up your environment to be usable even to start developing. I think these days, it’s … and maybe I’m a little bit insulated with working on very kind of specific open source projects, but that lead time has gone away a lot too, and I think that’s a big win too.

By shifting left capabilities, not only responsibilities, you enable developers to improve the system’s security [13:51]

Olimpiu Pop: Great. Thomas, you mentioned before the attack surface, and I cannot touch upon that because I spoke so many times about it, and my feeling is that nowadays everything is shifting left. If you attend conferences where you’re discussing security, it’s clear that security is moving left, shifting closer to the developer. Suppose you speak about data that’s shifting left as well. Pretty much now we move a lot of the things that used to be in the outer loop closer to the inner loop so that you can move faster. From all the tools that you brought together in your presentation, what do you think it helps a lot into improving the security of the applications?

Thomas Vitale: Yes, the whole shift left movement, of course, it’s really important. I feel like in the past few years we realized that too often we applied it wrong, meaning that we shift left the responsibility of lots of critical shared cross-cutting concern to each application developer, and that proved to be unsuccessful in many cases. So, we realized that we don’t want to push left responsibility, but we want to push left capabilities. And a term that recently got popular is shifting down.

So, basically we want to have a platform that provides these services to application development teams. So, sort of shifting down the platform that provides these capabilities, hopefully in a self-service manner so that developers are enabled to do better in terms of security, in terms of observability, but without the additional burden of becoming experts in all those fields and learning too many tools that are not part of a normal application development task.

So, in terms of security, I feel like getting more easy access to services, for example, in terms of SBOMs, like keeping track of all the dependencies in a project, and I know both Quarkus and Spring Boot integrates with CycloneDX projects, so whenever you work with an application, you can auto-generate a software bill of materials so you can keep track of all the licenses and the dependencies that you have.

That’s key for vulnerability scanning. But if you have a platform that provides the services, you can also automatically, whenever you push and you change, get a report, find out right away if maybe you’re using a dependency with the wrong license or a license that is not allowed by your organization. There’s lots of attention these days around compliance and security, especially in the public sector. So, getting these capabilities already inside the inner loop … again as services, not as something that developers are responsible for … I think that would really improve the overall security posture of the software.

Olimpiu Pop: Okay.

Kevin Dubois: Yes, I definitely agree with that. I think also integrations into your development cycle itself, right? Into your IDE when you’re importing, when you’re adding a new dependency to your code that you immediately get a notification like, “Hey, this dependency is not up-to-date”. Or you open a project or you start writing code and it is able to scan and say like, “Hey, you’re doing something that you aren’t supposed to do”. Those are the things that we mean by shift lift, right? It’s not like the developer gets all this extra stuff that they need to now work on. It’s just like, here’s a solution for you, instead of here’s a problem for you that you now need to solve as well, right?

Olimpiu Pop: This sounds more like … somehow, a lot of things appeared in the last couple of years. We just dropped things like flux, and we discussed the developer experience, the platform, and quick feedback loops and all the other stuff. But now, somehow listening to you guys, there is more togetherness in software development because people are working together as a team, but also as companies. And this is pretty much a depiction of that, what you mentioned, that probably you have a platform team that is focusing on all these kinds of things that are being built. And then on the other side, you have the tool that allows you to act quickly at the developer level.

Now, given that Kevin is in Brussels, how will the Cyber Resilience Act play with this? Because for a long period of time, people are terrified of what that might bring, but now, as Thomas was mentioning as well, you can generate your BOMs quickly directly from the toolset that you have, and that will allow you to act quickly in terms of other issues as well. So, any insights on those?

Kevin Dubois: Yes, so I’ve worked with customers here in Brussels, but it’s, of course, just not just in Brussels. It’s across Europe. It’s also in the US, depending on what kind of organization in the US. But they need to provide the origin of what kind of software. So, the software bill of material. They also need to make sure that in terms of the container images, what you’re using in those images comes from reputable sources, or that, at least, you can trace it back also that you signed those SBOMs in those containers so that you know during your build process it was really you who provided those things.

It’s not somebody who then snuck into your build pipeline and started doing some malicious stuff. And when we’ve started talking about this a few years ago, it was this huge kind of hurdle for organizations, like, how are we going to do all this? It’s like maybe there’s a tool to create SBOMs and maybe there’s a tool to sign images. There’s the project six store and different projects in that space. But to bring that all together was quite the challenge.

So, what we’ve seen, especially the last year, year and a half, that there’s different solutions. Red Hat provides a solution, of course, but other organizations and different products provide those solutions too to kind of automate that during your pipeline. So, really, it’s a relatively straightforward process, and I think we kind of have these regulations to thank for that because this was really needed, especially as everything becomes more automated and systems become more used for critical data that we’re sending around our countries and to different people, it’s important that we do really think about what is being used and how is it being used, and if we find that there’s an issue that we can quickly trace it back and then we can fix it. Not like a couple of years ago with Log4j, where it took organizations months, if not more, to find out where is it actually being used and then being able to finally track it back and fix it. So, I think that was also a good wake-up call for organizations. We need to automate security as much as everything else.

Olimpiu Pop: My take on this is that the industry is coming of age. We kind of realize that software is eating the world, but we have to take the responsibility that we need to, especially in these days. And, yes, I think it’s a good step forward that we have some legislation in place that forces a couple of the things to be done, especially that it’s really a lot of things are happening out there. I know, I think the last statistic was that around 90% of your application is actually built by somebody else because it’s open source. So, you are just importing stuff that you don’t actually know.

Kevin Dubois: And with AI probably even more so, right?

Olimpiu Pop: Exactly. Yes.

Kevin Dubois: You’re even having our-

Olimpiu Pop: You read-

Kevin Dubois: Yes.

How to integrate generative AI in your Java application and still play by the rules [21:32]

Olimpiu Pop: You read my mind. So, given that Thomas mentioned that he’s putting a lot of effort nowadays into getting AI into Spring and more, let’s say, traditional type of building applications, somehow it’s against the current because AI is big. And then we are discussing about frugality and we’re discussing about faster startup times and the quick loops, and then we are discussing about transparency. And AI is not always transparent

Kevin Dubois: Or green.

Olimpiu Pop: Exactly. So, how can you still have the AI experience put in the small Java box or virtual machine or whatever and still be efficient, Thomas?

Thomas Vitale: Yes, I guess most of the time these days when we talk about artificial intelligence, we talk about large language models and generative AI. That, of course, is just a part of AI. But what we’re doing these days, I guess we have two areas. One is AI assisted development, so using AI tools to help us develop software. But what frameworks like Spring AI or LangChain4j do is allowing us to integrate Java application with AI capabilities.

A great thing that I like about this most recent wave of AI is this consolidation of HTTP APIs in front of the model inference services, meaning that all of a sudden you don’t need to use Python anymore, but you can use your favorite programming language, the one you’re the most comfortable with or the most productive with, and just by coding an HTTP API, you can consume these services. I feel like that was one of the key thing that enabled, at scale, all developers suddenly to be able to consume these machine learning services. And I think that’s really a powerful concept.

So, from that point of view, considering Java is already established as an enterprise tech stack, it’s yet another integration. We have many other types of integration, but of course that comes with lots of challenges. The main one is we cannot rely on large language models 100%. They’re not deterministic. So, one thing is the integration part.

From a technical point of view, Java is really well-equipped to solve that. But the other part is if we want to really use this in production, how can we validate what it’s doing? As you said, it’s a black box. So, there’s a lot of research going on right now about explainable AI, like, okay, we have this machine learning model, we don’t know how it works. We don’t know what’s inside. And there’s lots of research happening just to introduce some more determinism as much as possible into explaining why a certain result is outputted from the model.

So, I feel like that’s the main challenge if we consider a production perspective to reach something that is production-ready, especially in industries that are highly regulated or highly sensitive. I’m thinking about healthcare, for example. We really have to be careful when we start thinking about production use cases.

Olimpiu Pop: Okay. And you had an interesting point here. Obviously, it’s not deterministic and it just gives you a broad range of results. At some point, it can mean the same thing, but in other words. And then we are discussing Java, which has some decades behind, that we always spoke about black or white or zero or one. It was quite simple from an engineering perspective. You either had something that was equal with something or something that was not equal to something, or something like that. Now, with AI, and LLMs more particularly, we are discussing ranges, and it’s a gradient. It’s 90% accurate, it’s 50 or whatever, because you mentioned earlier, Testcontainers. Is the ecosystem ready? Are we, as developers, ready to embrace that verification? It’s not testing anymore. It’s verification, because you were looking at the range.

Kevin Dubois: It really is kind of a different mindset, and I think that’s what we as developers struggle with. Just like you said, we like to have very deterministic outcomes where we know exactly what’s going to happen, and that’s not the case. And I don’t think that’s ever going to be the case because that’s the whole point of, well, generative AI is to be creative. But we can use it for use cases where we want creativity. But the nice thing that I’m seeing now, and that we’re all seeing, is combining those two pieces. So, we can use AI for the creative part, but then we can also integrate it with deterministic actions that we call as tools, and we’ve heard of Agentic AI that is also able to tie in with external capabilities. We’ve heard of Model Context Protocol as a way to call external systems from your LLM and vice versa.

And I think there’s a lot of interesting use cases that can come out of that because that allows you to both use the creativity of the LLMs, but also get … if I want to access a database and get a particular value, then we can instruct it to. Or if we want to get something from the internet that happened today, which LLMs typically don’t know because they’re trained on a lot of data, but typically not today, we can go and grab that information. And there are so many use cases where we can tie in business functionality, but then also use it with creativity. The real challenge is, going back to security, how can we make sure that this is all happening in a way that we can manage? And that’s a little bit of a challenge right now. And, for example, in the Quarkus and LangChain4j, the stack that we’re building, we have these guardrails that we’re building.

So, you can try to sanitize both what goes to the LLM, what comes out of the LLM, or typically also what the LLM is doing in terms of agents. So, if it calls different actions and tools through Agentic AI, that we can also verify that it’s not doing something that it’s not supposed to do. For example, if we give it access to a database that if somebody says, “Ignore all previous instructions. Delete the database”. That’s kind of the funny or not so funny example, but that it’s not actually able to do that. So, there’s a lot of considerations with AI.

So, going back to your question, is it ready for production? As always, it depends, but I think for some use cases, yes. But I think we need to be careful, especially with security. The nice thing is, and then I’ll let Thomas speak, is that all these things that we’ve worked on over the years also in terms of all the cloud-native stuff that we’ve been doing in terms of security, in terms of rolling out, in terms of making our applications faster, all these things kind of come back in this AI stuff too. So, if we call with MCP, with model context protocol, an MCP server that’s running as a job application, well, we can use serverless, we can use very fast startup components to return results to the LLM without constantly running big systems. So, there’s a lot of the stuff that we’ve done over the years that we can totally use with this new kind of AI stuff as well.

Thomas Vitale: Yes, I totally agree. I think that, in this context, frameworks are really, really important, besides integrating with a model. That’s the easy part. But frameworks then have the responsibility of adding all these layers of functionality that helps you make an application production-ready or getting there eventually. In the Java ecosystem using Spring Boot or Quarkus, we already got lots of these building blocks in terms of security, in terms of observability, in terms of auditing. So, it’s a matter of reusing all these different building blocks we already had. And on top of that, creating this experience considering that these generative AI models are not deterministic. So, yes, something like guardrails, verifying input-output, and evaluation frameworks. I know both Spring AI and LangChain4j have these features.

But probably the most challenging part I see is what you also mentioned, Kevin, we need to change our way of working with it because we are used to something that works all the time or doesn’t work. Like before, integrating these large language models, we never had discussions like, “Oh, this application works 90% of the time. Okay, let’s ship it”. No, that was never acceptable. So, either it works or we don’t ship it because then it’s not ready. But here we have to change the way we think about it, even the way we test it. Testing drastically changes. So, how do we test these applications? And that’s also why, since we cannot rely on these models, we need these additional strategies that also influence the user experience, keeping humans in the loop. We cannot automate everything, so we need to design these steps where we ask the end user to validate some of the content or some of the answers from the models before we move on, to bring a bit of safety and more determinism into the whole AI workflow.

Olimpiu Pop: Interesting enough, I had a similar conversation, or more targeted on the generative AI space and what that means, with Jade Abbott. She’s the CTO of Lelapa AI, and they are focusing a lot on generative AI into spaces that are not exactly that powerful in terms of infrastructure, mainly in Africa. And what she was mentioning is that she had two parts of her brain. One of them is an engineering part of the brain that’s zero is zero, and one is one, and then it’s the machine learning expert where you have a spectrum, a statistic. It’s zero point something and something like that. And then she was mentioning the definition of the bug. What’s an engineering bug? Where one should be equal to one, while in the machine learning space, the AI space, it’s more of a gradient. It’s 50% a bug, it’s 55% of the bug. And I think that’s something that we have to understand.

Do we have solutions for running LLMs locally? [32:11]

And going back to the point that we had earlier, Thomas, when we were saying that long are the days when we had to use … I don’t know, HSQL or H2 SQL for our testing purposes. The L in LLM, the first L in LLM stands for large. Most our machines are not able to host a large language model for the inner loop. Do we have an option or how should we proceed with that?

Thomas Vitale: That’s a very good question. I think maybe two years ago when this whole new trend started, it was indeed a problem. But I feel like right now it’s less of a problem. More and more models are getting smaller, while providing maybe half the size they were last year, providing the same performance though. So, it’s becoming easier and easier to run models on your local machine. And I really think that that’s going to be an ever bigger trend in the future. Running models on your machine, that solves several issues. One is the developer experience, of course, but also in terms of being green and the energy consumption. We are not relying on Cloud resources explicitly for that. I’m already using my machine, I’m already using the resources, so let’s use it also for that. So, I don’t need extra computation power, extra infrastructure deployed somewhere.

I was on a plane recently, no internet connection, and I was just coding. I could use local models thanks to Ollama, both for AI assisted development, but also integrating my application with AI capabilities. So, I feel like we are already there, but I think it is only going to get better from here on. That would be my forecast. I don’t know if you agree, Kevin?

Kevin Dubois: Yes, no, absolutely. I had the exact same experience a couple of weeks ago when I was on a long flight and I really was counting on there being internet. And then you get on the plane and the first thing they say like, “Oh, sorry, the WiFi doesn’t work this time”. And it’s like, ugh. And just like you, I was like, “Oh, wait, I can just use an LLM on my local machine”. And I was able to do most of my work. Even with the code editor, I think I was using Continue or something, and I think I was using Podman AI Lab, but kind of the same inference, running a small model on my local machine. And it worked relatively well. The only kind of funny thing is that I was testing it out and I was like, “Who’s Kevin Dubois?” And it answered saying like, “Oh, he’s a famous soccer player from Belgium”. So, at least it got the Belgium part.

But going back to what you were saying, Thomas, about verifying what the LLM is actually producing. So, some models you can actually ask them, where did you get that source? And so I asked that and it was very convincingly gave me a URL for this famous Belgian soccer player called Kevin Dubois. And I’m like, wow, okay, maybe this person does exist, but I don’t have internet so I can’t really verify. And then when I landed, I looked up the link and it didn’t work. So, it’s very convincingly trying to tell you about facts that sometimes aren’t. But aside from that, no, it was working really well to generate some stuff. And then, of course, in terms of code, you need to hold its hand because it’s still kind of a very junior developer, but it is fascinating to see it all work. Yes.

Olimpiu Pop: I didn’t intend to ask you this, but given that the vibe led us there, I have to ask-

Kevin Dubois: Did you say vibe on purpose here?

How to ensure the quality of the code merged in the Gen AI frenzy era [35:51]

Olimpiu Pop: Yes, yes, of course. You caught me. So, we spoke already about AI and the way how we integrate it in the application and that we have to be careful with. The other side is that, as you mentioned, generative AI is very useful in terms of generating code. And now it’s so easy to just be led and just let it go, especially when you have hard deadlines and the things that you have to make sure that they are getting to production. But how can we make sure that we use all the tools that we have built until now as an industry to make sure that what we put together is really at the quality standard that we would like, so that we avoid all the things that we mentioned before, besides the quality issues, the supply chain attacks, because that’s quite easy to push with generative AI.

Thomas Vitale: Oh, that’s a really good question. You want to go first, Kevin?

Kevin Dubois: Sure. I mean, I think the role of the more senior developer is even more important than it already was. I think it already was important, as a mentor. In this case, the senior developers have lived the pre-generative AI world, so they know what to look for and they know … well, ideally, at least what the gotchas are, and they’ve seen our evolution towards better automation and making sure that all the boxes are checked for deploying to production.

What I am a little bit worried about is the junior developers that are coming on, like, oh, yes, they’re very good at using these tools, which is great. I think that’s a really big asset, but I think it’s up to the senior developers to also take them under their wing to make sure that they’re still doing it the right way and that they don’t just rely on what the code is that’s being generated by these tools because they tend to be … and maybe it’ll get better, but they still tend to be quite verbose and aren’t necessarily using the latest kind of tools or programming paradigms that are out there because they’ve been trained on previous data, and then they kind of train themselves on that data that they’re also producing.

So, there’s a significant role for the senior developers, and I think there’s maybe a gap between the junior developers and then getting to that senior level that we might see. But, yes, we’ll see

Thomas Vitale: Yes, in terms of tooling, but also in practices, I’m a bit worried that we tend to forget about them too much. Also, from a organization management perspective, lots of money has been assigned now to AI. Everybody’s using AI. More and more people are encouraged or even forced sometimes to use AI. But we have a wide variety of tools already and practices established over years and years of software engineering that I feel like sometimes we are forgetting.

For example, we know about test-driven development, starting from a small test in order to validate the design and then move to the implementation. And now what we’re using AI for is I implement the business logic and then I generate the tests because writing test is boring. And then, of course, they all eventually pass after a few rounds. But then are we testing the requirements? So, are we just testing that implementation the model came up with is correct?

I see that happening more and more. And I feel like we should talk more about doing the opposite. I write a test, and then AI writes the implementation, maybe? But, in general, I hope we rediscover the existing tools and we don’t forget about them.

For example, in the cloud-native space, we talk a lot about platforms and one feature, a very convenient feature about platform … very good platform at least … is this capability of bootstrapping new project. Think about something like Backstage, a portal where a developer can go. Maybe I want to build a new web application and then I choose a certain template and I bootstrap a new project with all the guidelines, all the compliance, all the rules that that specific organization has decided to be compliant with, and I can immediately run the project. Maybe I’m using Quarkus or Spring Boot, so I just do git clone and run. Don’t have to install anything else.

Instead of doing that and maybe becoming better at leveraging those tools, those cloud-native integrations … so, also to auto-generate pipelines based on something deterministic … there’s this trend of let’s just have the AI model generate everything from scratch every time. Okay. It might work.

At least for demos, it looks quite impressive. But when we consider enterprises, when you consider production-ready scenarios, I’m a strongly supporter of still maintaining a certain focus and also financing the advancement of all the existing tools and practices that we know of that result in good software engineering practices because we have lots of tools available for doing … even security checking, for example, for migrating a project, upgrading to new version, there are tools like OpenRewrite, there’s Konveyor. We have tools that can deterministically, and in an auditable way, do lots of these tasks. And sometimes I feel like we’re skipping … like, we know about automation, we’ve been doing automation for so many years, but now we just forget all about it. And any automation is using large language models, but that’s not always a good answer.

So, let’s not forget about all the great things that happened in the past few years. Let’s just use large language models as an additional tool in our toolbox, not just as the only tool in our toolbox.

Olimpiu Pop: Yes, that’s my feeling as well is that now we have the tendency of just going and using whatever large language model you have close by to do basic stuff. That until yesterday, you did it by just thinking about it. For instance, I was just, one example is I know I had the phrase and I was curious about how many words are in that one. So, I just dropped it in ChatGPT, and obviously that’s too big a hammer to do it, while you can do it with wc and just have it in a very plain terminal window and just get all the information that you need. So, I think that’s important. And my feeling is it also, it’s pretty much how the search engines, or Google, used to be when I stopped using bookmarks because I had a couple of words, keywords in my mind, and I was just easily finding that thing. But it’s very expensive because it’s pointless to just go all the way there.

Does the sovereign cloud tendency change anything? [42:33]

But basically what you both are saying is you have to be focused on best practices that we learned through the experience of other generations and make sure that our focus is close to that one. So, rather than treating the generative AI as a new colleague in the team that has a different status, it has to go to the same quality gateways as normally, like proper code review, all the checks that you normally have when just pushing the code through production. And again, there is another topic that I have in mind that I wasn’t about to ask you, but, well, it just pops, so I have to, because you’re discussing about the cloud native. The tendency nowadays in cloud is sovereign cloud. So, how do you think all the practices that we develop in the cloud-native ecosystem will change, if they’ll change in any way, with this new tendency?

Kevin Dubois: Yes. Do you want to explain sovereign cloud for maybe the listeners that don’t know it? I mean-

Olimpiu Pop: Well, go ahead. You’re closer to the source than I am, so go ahead.

Kevin Dubois: I mean, hopefully I get it right. But it is basically meaning that we want to make sure that our clouds aren’t running on some cloud provider, that if something happens, Lord forbid, especially in the US, that we as European consumers of these clouds, it doesn’t mean that our data is going to be accessible or, even worse, not accessible by us anymore. I mean, I think that’s kind of the idea.

And I think that’s exactly where our work in the cloud-native space is so great because it’s all open source, or at least most of it is. And it’s sharing these tools so that if something happens, I mean, let’s say Kubernetes is still there, right? The same deployment techniques that we’re using to deploy to one cloud provider we can use for another cloud provider. So, if we build our systems the right way without lock-in to specific providers, it means that we can move our systems around. And then hopefully we also are keeping that in mind for our data and not just go all in with one provider where if something happens, whether it is for regulation purposes or for any kind of other reason, that still remains possible. Right?

Thomas Vitale: Yes, I agree. I feel like all these cloud-native technologies that over the past few years became more and more popular, they are actually key enabler for this strategy here because you can run Kubernetes across many different providers, infrastructure, on-premises, on Edge, in the public cloud from whatever country, and lots of tooling around Kubernetes also working that way. They’re open source. They are not vendor-specific. So, of course, each cloud provider adds lots of value in terms of convenience, in terms of services.

But all the core infrastructure and platform and tools around development, delivery, deployment, observability, they are all open source projects. There are open specs. I’m thinking of, for example, about OpenTelemetry where you can use these common APIs supported by all observability vendors, so you’re not locked on a specific vendor, but you can move. So, the impact on the business investing in cloud-native technologies, it provides lots and lots of benefits for many different reasons, but not specifically for reaching this digital sovereignty. I think it’s a key enabler.

Olimpiu Pop: Okay, thank you.

Kevin Dubois: Yes.

Olimpiu Pop: So, basically what we are saying is everything that we put together as an industry, basically how the Java ecosystem worked for a long period of time where you had a definition of an interface and then you had the implementation behind the scenes that allowed you to move from one side to another is happening now in the cloud-native space and all the things that would build around the CNCF, the Cloud-Native Community Foundation, allows you to be quite independent from a particular vendor that will allow you to move from one side to another and remain in an arm’s reach, so to say, with these points. Okay. Is there anything else that I should have asked, but I didn’t?

Thomas Vitale: And maybe just a comment on what we just talked about. I think that whole conversation around cloud-native technology also include the new developments in AI. So, lots of machine learning models are available as open source. I’m thinking about Mistral AI from France. They publish their models as open source, Apache to license. So, you can run them on your local machine on premises, or you can use the cloud platform, but also from that perspective, that ensures portability. So, you’re not locked on a specific platform or infrastructure vendor, but the combination of open source and open specs, I believe, is a key combination for this entire discussion.

Kevin Dubois: Yes, exactly. And there’s a variation of open source in the AI space right now too, right?

Thomas Vitale: Oh, yes.

Kevin Dubois: Because there’s some models that are like, “Oh, yes, it’s an open source model. You just need to agree to our license and then make sure that you agree to where you can use it and how you can deploy it”. Yes, of course, we need to be a little bit careful with that. And then open source, in my opinion, also means that we have visibility into the sources. So, what was it trained on? What were the weights that it was using? So, that’s where we see some questionable definitions of open source. And I think that’s, especially in terms of the sovereignty, it’s going to be a really important topic.

So, a little shout-out to IBM because they’re really working on models that are completely open, where they disclose the sources that they’re trained on and the weights with their Granite models. And I know that there’s quite a few organizations that are really looking for that kind of real open source models, because, otherwise, well, if we use our models and it generates some stuff that it shouldn’t, because it’s trained on proprietary data, an organization doesn’t want that to happen. They don’t want to be sued because they’re now disclosing information that was trained on, let’s say, Thomas’s books that … that it shouldn’t do that, right? So, I think we’re going to see a lot of-

Thomas Vitale: Well, that happened but it’s fine.

Kevin Dubois: Yes, exactly.

Thomas Vitale: But actually that lead us back to … we were talking about supply chain security, and this is part of that discussion. For example, CycloneDX has a spec also for generating this or providing bill of materials, also, for machine learning models, because it’s really important to know we need transparency. We need to know the data set that has been used for training. We need to know all the ways … It’s not easy right now. There’s lots of … well, let’s call them discussions and arguments around it, but for organizations, it’s not easy at the moment to find the specific models that give you that same safety that you rely on a standard open source project because it’s different. It’s not just the software. There are so many moving parts and there’s still not enough clarity. So, I see that as a bit of a challenge we need to overcome soon if we want to move forward in production use cases.

Olimpiu Pop: Yes, that’s true. And what I was thinking … well, closing the gap with what we discussed earlier, that we have more than the BOMs. And in terms of what you actually get, we have also six store when you have a signing for your libraries, and then there also the reproducibility part of it. And I think that’s probably the next step with the open source AI. It’s now getting a lot of momentum. You see Hugging Face has a growing number of millions of models available, but what’s actually missing are these parts. What data was it trained on? What are the parameters and all the other details? That’s probably the next step. And somehow I’m calmer now with the European legislation that is coming together because it creates some safety nets from that angle.

Thank you, guys. Thank you for your time.

Thomas Vitale: Thank you.

Kevin Dubois: Thank you.

Mentioned: