Key Takeaways
- Software architecture is about decisions, but you can’t make decisions about a black box. AI-generated code is largely a black-box; it could be understood but no one has time to do so.
- The only way to evaluate the behavior of this AI-generated “black box” is through experimentation. As a result, software architecture in the age of AI needs to become a primarily empirical approach.
- The art of architecting is knowing which QARs most affect the architecture of the system. Validating software architecture empirically involves architectural testing focused on validating QARs.
- AI will cause a shift in architectural focus from deciding how the system will work and how it is structured to how the system’s architectural qualities will be validated.
- When interfacing with the AI, teams need to be far more articulate about their trade-offs and reasoning about them so the AI can generate potential solutions that satisfy these trade-offs.
AI is turning out to be a powerful tool for developing software. In a prior article, we articulated some ways that AI can be useful for teams in creating their software architecture. Inevitably, teams will be tempted to go beyond using AI as an assistant that helps them brainstorm alternatives to using it to generate code that implements a Minimum Viable Architecture (MVA). When this happens, the work of architecting may change substantially.
Software architecture is all about decisions, but AI-generated code is a black box
“Any sufficiently advanced technology is indistinguishable from magic”. – Arthur C. Clarke
AI’s code generation capabilities can seem almost magical, at times, but along with these capabilities comes a conundrum: you really can’t see or understand why the AI generated the code that it did; it’s just the way the model works. Teams can use AI to generate code for their MVP that, implicitly, makes decisions about their MVA, but there are several architectural issues that teams need to consider when they do this:
- While the AI generates an MVP, teams can’t control the architectural decisions that the AI made. They might be able to query the AI on some of the decisions, but many decisions will remain opaque because the AI does not understand why the code that it learned from did what it did. This is similar to the framework problem we discussed in an earlier article: frameworks make decisions for you, but you don’t always know what they are.
- From the perspective of the development team, AI-generated code is largely a black-box; even if it could be understood, no one has time to do so. Software development teams are under intense time pressure. They turn to AI to partially relieve this pressure, but in doing so they also increase the expectations of their business sponsors regarding productivity. As a result, development teams never have the time to understand the architectural decisions that the AI has made in producing the code.
- In a sense, AI is a factory for producing technical debt, and like virtually all technical debt that we have encountered, it is never “repaid” until there is a catastrophe. AI-generated code is not intended to be maintained, per se, but can only be replaced by more AI-generated code. This leads to an open question about the sustainability of the system; teams hope that future AI coding engines will be able to replace code with better, more sustainable code.
An example that encompasses all three of these challenges is the case where AI-generated code must interface with existing systems in a way that satisfies QARs, e.g. security requirements. For the foreseeable future, AI-generated code will always need to interface with existing systems, usually through an API. Development teams need to ensure that the QARs for the entire system-of-systems are still met.
The only way to evaluate the behavior of this AI-generated black box is through experimentation
As we explored in an earlier article, teams have three questions they need to answer with respect to their architectures; using AI does not change this, although it can help them to evaluate the questions more quickly:
- The most costly decision, and the one to consider first, is building a product that isn’t worth building. Using AI to generate (wholly or partially) a solution can help a team by letting them test an MVP with customers sooner, and with less effort.
- If the product is worth building, the next most costly decision is building something that can’t perform sufficiently or scale up to satisfy its business case. Using AI can also help the team to arrive more quickly at a design that they can empirically test, but if it doesn’t scale or perform, the team’s alternative is to simply generate something else. If they reach a point where no generated solution meets the QARs they will have to build the solution by hand, but they have lost time evaluating rejected solutions. By this point they may have blown the business case.
- Once these questions are satisfied, the next most important decisions relate to lifetime cost – the choices that make the system maintainable and supportable over its lifetime. This is where AI-generated solutions become most problematic. Like all code generators we have worked with, AI-generated code is not meant to be maintained; if it is wrong or fails, it has to be generated again using new prompts (or the same prompts with a new model).
Empirically testing the AI-generated system against its QARs may be the only way to really understand the suitability of the AI-generated architecture. The art of architecting is knowing which QARs most affect the architecture of the system. Teams never have enough time to test everything, so knowing where to focus is essential.
AI shifts the architectural focus to how the architecture will be validated
Teams need to develop new skills and insights to meet this challenge. Some traditional techniques like architectural design reviews, code reviews and inspections as well as security reviews are simply impractical and ineffective given the volume of AI-generated code in the system. Using AI to review code is a possible answer, but since the AI-generated code was never meant to be directly maintained, code reviews are not particularly useful for AI-generated code.
As a result, the nature of the work of architecting will shift from up-front design work to empirical evaluation of QARs, i.e. acceptance testing of the MVA. As part of this shift, the development team will help the business sponsors figure out how to test/evaluate the MVP. In response, development teams need to get a lot better at empirically testing the architecture of the system. Here is a partial list of the techniques that can be used for that purpose:
- Performance and scalability testing, focusing on how well the system meets its QARs.
- Usability testing, to evaluate how effectively users can complete specific tasks to ensure the system can be used easily and productively.
- Change cases, including both architectural change cases (change cases that directly impact the QARs) and “functional” change cases that indirectly impact QARs.
- Ethical hacking, probing the system, using the same tools as the hackers do, to uncover security vulnerabilities before malicious attackers can exploit them.
- Chaos Monkey – an open-source software tool developed by Netflix that helps identify and fix vulnerabilities. It randomly terminates virtual machine instances and services in a production environment to test system resilience.
Testing a system that includes AI-generated code becomes even more important, and needs to shift from functional testing to architectural testing. As part of this effort, the development team will need to identify and implement tools that automate these techniques as much as possible.
Software Architecture is still about decisions and trade-offs
Using AI to generate the MVA does not change the basic fact about software architecture: teams still have to make decisions about trade-offs. The team needs to know what trade-offs it may need to make, and they need to articulate those in the prompts to the AI. The AI then works as a very clever search engine to find possible solutions that might address the trade-offs. As noted above, these still need to be evaluated empirically, but it does save the team some time in coming up with possible solutions.
We summarize this approach as “caveat prompter” since you have to understand the problem and the trade-offs, or you won’t be able to provide enough information to the AI to generate a good result. This means that teams need to be explicit about trade-offs and alternatives when they create the text for the AI prompts in order for the AI to manifest those trade-offs in the code it generates
Conclusion
Does AI-generated code bring an end to software architecture? No; teams still need to make architectural decisions and trade-offs, but they need to be far more articulate about their trade-offs and reasoning about them so they can supply the AI with these ideas in the prompts.
Like any technology, however, AI solves some problems but also creates new ones. With a successful MVP and an MVA depending on AI-generated code, the development team may be playing a dangerous game – they may have delivered a system that may break at some point and they may not be able to fix it. To make matters worse, the quality of AI generated code may degrade over time, with newer models more prone to silent but deadly failure modes, as they are trained on poorer quality code (often itself AI-generated) than the older models. This would make it even more difficult for the team to improve a system depending on AI-generated code over time.
In addition to shifting their software architectural focus toward empirical validation, teams will need to think about maintainability in a different way: will they be able to support the system in the future when the AI may work differently, or perhaps not at all?
