Key Takeaways
- As large language models (LLMs) have become widely adopted, AI-related innovation is now focusing on finely-tuned small language models and agentic AI.
- Retrieval-augmented generation (RAG) is being adopted as a common technique to improve the results from LLMs. Architects are designing systems so they can more easily accommodate RAG.
- Architects need to consider AI-assisted development tools, making sure they increase efficiency without decreasing quality. They also need to be aware of how citizen developers will use these tools, replacing low-code solutions.
- Architects continue to explore ways to reduce the carbon footprint of software. Cloud cost reductions are a reasonable proxy for efficiency, but maximizing the use of renewable energy is more challenging.
- Designing systems around the people who build and maintain them is gaining adoption. Decentralized decision-making is emerging as a way to eliminate architects as bottlenecks.
Each year, InfoQ editors and industry experts meet to discuss the latest trends we’re observing in software architecture and design. We categorize those trends using the Crossing the Chasm model created by Geoffrey A. Moore. Based on our collective opinion, the trends identified as innovator or early adopter are the primary focus for InfoQ editorial staff for the next year. These are also the topics that InfoQ readers should pay attention to as they may provide ideas for your solutions. In addition to this report and the trends graph, the accompanying podcast features a panel discussion that digs deeper into some of the trends.
Because software architecture decisions always come down to trade-offs, there is never one right way to solve all challenges. For this reason, there has always been a healthy debate among the editors for when a trend should move along the adoption curve. Much like those “it depends” choices when architecting a system, sometimes this comes down to subjective opinions over quantifiable data.
One aspect that helps determine where a technology or trend fits is how stable it is. Typically, innovative ideas are rapidly evolving, and the lack of maturity and stability require extra investment to make them successful. When we decide a trend has crossed the chasm from early adopters to early majority, that signals more companies should be able to adopt it, or at least consider if it makes sense for their situation.
AI trends for architects
The past year has seen AI-related topics evolving in many ways, and that will continue to be a focus of innovation across the industry. Large language models (LLMs) are now prevalent enough that they not only crossed the chasm, they jumped from early adopter to late majority. LLMs are the most frequently used part of any company’s pitch that they are using AI. However, with that ubiquity comes a lack of clarity about what LLMs are being used for, and even if they are the correct tool for the job. Ironically, that reveals another way to identify when a new technology has crossed the chasm–it starts being used in inappropriate ways.
The InfoQ AI, ML and Data Engineering Trends Report goes into more detail about some of these topics. In this report on architecture and design trends, we’re highlighting some of the major trends that software architects need to be aware of. Think of these as boxes on a context or component diagram. While not necessarily qualified to personally implement every component and subsystem, an architect needs to understand how an AI element relates to other parts of their system. What are the inputs and outputs? How can we measure performance, scalability, cost, and other cross-functional requirements?
Follow the AI, ML & Data Engineering topic on InfoQ for more information.
Agentic AI – Innovator
Looking past LLMs, two areas of innovation for AI that software architects should pay attention to are agentic AI and small language models (SLMs). Previously referred to as “AI agents,” the idea of agentic AI is to design AI models that can accomplish tasks autonomously. In some cases, multiple agents can collaborate to achieve greater results. In traditional software, we may see patterns of orchestration or choreography to manage workflows. An agentic approach may start with agents performing specific tasks, and could evolve into a supervisor approach where the AI determines which steps to follow in the business process. This approach remains an innovator trend because there is a sizable gap for companies to trust nondeterministic software to make important decisions.
Architects designing for agentic workflows can follow some of the patterns of microservices, with each agent having clearly defined boundaries and interaction patterns. These designs lead to better outcomes because the quality of responses and actions between the agents can be observed and tuned. Maintainability also improves, with individual agent modules being upgradable or replaceable as newer models become available. To ensure the agent is responding appropriately, ongoing testing must be performed as either the agent or its possible actions evolve.
Watch Shruti Bhat’s QCon San Francisco keynote: Pioneering the Future: Advancing Infrastructure for AI Agents for more information.
Small language models (SLMs) – Innovator
Architects are looking at SLMs as one option to utilize the features of LLMs, while also improving on some of their drawbacks. SLMs are usually more specialized than LLMs, and are therefore able to outperform LLMs for certain tasks. This specialization also means they are easier and cheaper to train, allowing more companies to create specialized models. Their smaller size also leads to lower operational costs and a smaller carbon footprint, as well as additional deployment options. Instead of accessing an LLM via a cloud-hosted API, SLMs can be deployed on self-hosted hardware, or on edge computers, thereby eliminating network traffic latency and increasing data security.
Follow the Large language model topic on InfoQ (most SLM content relates to LLMs) for more information.
Retrieval-augmented generation (RAG) – Early adopter
The most common technique for getting higher quality results from an LLM is to add RAG. Although RAG has quickly become a de facto approach, effective implementation still requires effort. Software architects are adapting their systems to provide data that can more easily be consumed for RAG use cases. In the future, we may see systems being designed with data expected to be consumed for RAG purposes. This relates to the broader trend of data-driven architecture.
Follow the Retrieval-augmented generation topic on InfoQ for more information.
AI-assisted development – Early majority
Previously, InfoQ tracked low-code/no-code as an architectural trend because well-defined APIs provided extensibility to citizen developers. We decided to replace that trend with AI-assisted development because the use cases have significant overlap. While many professional software engineers are using AI tools to help write code, the use by citizen programmers is one area requiring architects’ attention. When designing a system to facilitate low-code development, extra care was usually taken to ensure the APIs were safe to use. AI coding assistants have further reduced the barrier to getting started, and in some ways make it easy to use APIs that may not have been intended for low-code solutions or by non-professional developers. Although AI-assisted development may be widespread, the fact that it arrived so quickly may mean architects are behind the curve when it comes to designing systems that are prepared for the code being written to access them.
Architects and engineers are also concerned about the quality of code produced by AI tools, even in the hands of professional developers. Because the output depends on the quality of the prompt, architects are finding ways to provide good prompts which help ensure coding and architectural guidelines are upheld. Tooling is emerging to help shape AI-assisted development, but is not yet at the level of other standards for coding styles such as linting or EditorConfig.
Green software – Innovators
Carbon-efficient and carbon-aware software continues to be an innovator trend to watch. Many companies focus on reducing cloud hosting costs, which can be a good proxy for reducing energy consumption. While this reduction is a useful benefit, there is far less focus on driving down the carbon footprint of a software system. Next-level thinking requires architects to consider both where and when their software runs to take advantage of renewable energy.
Time and location are both important factors in making software more green because so much depends on how electricity is generated for data centers. “Follow the sun” is generally appropriate when trying to maximize solar power. However, running servers closer to capacity is generally more energy efficient than leaving them in an idle state, so it may be better to run some tasks after hours, when demand is lower and capacity is already provisioned. Network traffic can also be a significant consumer of energy, so data should be processed locally whenever possible. These few examples demonstrate the complexity surrounding green software, and show why there will continue to be innovation.
Follow the green software and sustainable computing topics on InfoQ and listen to a podcast with the authors of Building Green Software for more information.
Privacy engineering – Innovators
Privacy engineering was added to the trends graph in 2024 to highlight companies that are thinking about privacy as a primary feature, rather than something they are implementing in reaction to regulations or other incidents. In some ways, the emergence of AI is making a stronger case for considering privacy up front. Before implementing an LLM, architects are asking what data will be transmitted over the network, will it be used to train future models, and does the usage conform to previously approved terms of use.
Follow the privacy topic on InfoQ for more information.
Socio-technical architecture – Early adopter
The evolution of the role of the architect and how architecture is practiced is always a subject to pay attention to. Complex software systems need to be designed around the people who will build, support, and evolve the software. The “shift-left” movement is one way this is happening, with various aspects of the software lifecycle being addressed earlier in the process. This should not be construed as a move to waterfall project management and big design up front. Rather, architects are discussing concerns with stakeholders earlier, and designing systems that are able to evolve and avoid last-minute rework to accommodate everyone’s needs.
Innovative architects are also trying to be less of a bottleneck during the decision making process. Rather than architects being the sole decision makers, architects can help facilitate architecture decisions that are made by other teammates. By providing advice and helping others navigate the decision making process, teams are able to move faster, make decisions that are more contextually appropriate, and have more understanding and confidence in the design.
How teams build and release software contributes to what they are able to build. For that reason, platform architecture is something architects need to consider. Engineering platforms have transitioned from bespoke solutions to almost a “platform as a product” commodity. Applying a socio-technical lens when evaluating build versus buy decisions ensures architects and other stakeholders consider how such solutions will benefit the people who will be writing, deploying, and maintaining software.
