Renovate to Innovate: Fundamentals of Transforming Legacy Architecture

This article is based on a talk I gave at QCon San Francisco in November 2024. In this talk, I explored the inevitability of legacy systems in successful companies and the importance of transforming legacy systems to accelerate innovation.

I discussed various strategies to tackle such technical renovation initiatives, like evolutionary architecture, deprecation-driven development, and intentional organization design. The goal of this talk was to provide actionable insights on modernizing legacy systems to ensure continued success in the future.

Understanding Legacy Systems

Successful companies go through different phases of growth, and each phase requires their software systems to evolve differently to meet the business’s needs. Software systems that are adequate for validating product-market fit in a company’s early stages may not be capable of supporting the business in the next phase of rapid growth. The exponential growth phase requires drastically different software capabilities than earlier phases. Successful companies, like the ones that live to see exponential growth, outgrow their software systems one way or the other.

Legacy systems are a byproduct of success. However, they tend to have a negative reputation. The term “legacy” can stir up strong emotions, and understandably so. We often associate legacy systems with technical debt, challenging migrations, high maintenance costs, and a less-than-ideal developer experience. To ensure your company’s long-term success, it is crucial to develop the capability to update and improve these legacy systems. Future success hinges on your ability to prevent legacy systems from hindering your company’s growth.

What Defines a Legacy System?

The term “legacy” is quite overloaded. Depending on the context, it can mean anything from outdated, unsupported technology to untested code. While these are common associations, in this article, I deem a software system legacy if it can no longer reliably meet a business’s evolving needs. Let’s review some symptoms of legacy systems.

Substantial Complexity

Legacy systems often exhibit complexity in one or more of the following dimensions: organizational, operational, and cognitive. Organizational complexity occurs when collaboration across numerous teams slows engineers down due to the need for extensive coordination. Operational complexity arises when a lack of automation, testing, monitoring, and observability results in high operational costs. Cognitive complexity develops when institutional knowledge accumulates, documentation becomes outdated, or key personnel leave the company.

Drop in Innovation Velocity

Why does complexity matter, you ask? As complexity increases, innovation velocity decreases. There is a direct correlation between complexity and innovation velocity.

Product and project managers expect productivity to scale linearly, but engineers know better. It is a symptom of a legacy system when the reality of how long it takes to innovate or build a new feature far exceeds expectations.

Degraded quality of Experience

Quality of Experience(QoE) measures the overall satisfaction of end-users when they interact with a system. Poor user experience, such as slow page loads, high error rates, and poor reliability, impacts the business. Amazon has an infamous study that suggests every 100-millisecond increase in latency impacts their sales by 1%. A dip in the quality of experience despite best efforts to optimize user experience is a symptom of a legacy system.

Why Do Systems Become Legacy?

Software systems become legacy for several reasons. The most apparent is the rapid pace at which technology advances. Systems that were once state-of-the-art quickly fall behind modern industry standards as technology choices become outdated within just a few years. Beyond this obvious reason, there are two schools of thought that further explain the degradation of software.

Bit Rot Theory

The Bit Rot Theory suggests that software gradually deteriorates over time due to incremental changes either within itself or in its environment. Examples of bit rot include unused code paths and code duplication. Even seemingly innocuous issues, such as inadequate documentation or the loss of institutional knowledge, can contribute to this degradation. While, in theory, good software engineering practices can mitigate bit rot, in practice, it often accumulates unnoticed until it becomes a significant hurdle.

The Law of Architectural Entropy

The Law of Architectural Entropy states that software systems lose their integrity when features are added without much consideration of the original architecture. The primary driving factor for architectural entropy is the real and sometimes unintentional tradeoffs that engineers make in order to deliver results faster or meet deadlines.

Imagine the growth of a successful e-commerce company. In the early stages, they’re focused on establishing a thriving business. Evolving their architecture to be perfect is just not a priority. In fact, changes to the architecture are driven by business needs. In this example, every new feature is added to the existing monolith, steadily increasing the architectural entropy.

In reality, software systems are impacted by all these factors and beyond, which is why outdated and legacy systems are more prevalent than we might prefer. Given the commonality of legacy systems, we must ask ourselves: do we take consistent and proactive steps to renovate them? Unfortunately, the answer is often no.

The inevitability of software degradation and the lack of proactive action to counter it result in systems that are challenging to maintain, comprehend, and extend. These systems can significantly hinder your organization’s growth and success, and sometimes, the only path forward is Technical Renovation.

What Is Technical Renovation?

I define technical renovation as the process of upgrading or replacing outdated systems to enhance software capabilities. While it is often confused with refactoring, there is a crucial difference between the two. Refactoring is akin to reorganizing a closet – rearranging items to optimize for space while keeping the larger architecture intact. It enhances the existing structure and improves maintainability without altering functionality. In contrast, renovation is like transforming a regular closet into a walk-in wardrobe. It is a more extensive endeavor that introduces new capabilities and sets up the system to meet future business needs that it could not otherwise have met. Renovation involves re-architecting systems, updating frameworks, and modernizing infrastructure to align with evolving business needs.

Although this discussion focuses on renovation, it is important to recognize that refactoring is vital for maintaining system health. Refactoring often reduces technical debt and aids maintainability. However, when systems reach a point where minor improvements are insufficient, a full renovation becomes necessary. Understanding when to refactor and when to renovate is essential to prevent legacy systems from impeding progress. Let’s look at some scenarios that necessitate a technical renovation where just refactoring does not cut it:

• When business requirements undergo a drastic change, such as Netflix’s transition from DVD rentals to streaming services, technical renovation becomes unavoidable. The technical capabilities required to deliver DVDs are very different from those required to stream video on demand. This is an example of a scenario where technical renovation is driven by changes in business needs.
• When the system architecture needs a significant evolution, such as Netflix moving from Monolithic to Service Oriented Architecture, necessitated a technical renovation. As Netflix expanded streaming services globally, it became increasingly difficult to scale and deploy new features quickly with a Monolithic architecture, transitioning to Microservices allowed for independent deployment and scaling of services. This is an example of a scenario where technical renovation is driven by an architecture need.
• Technical debt occurs when you borrow from the future to make a tradeoff for the present. It is a real tradeoff that engineers have to deal with to meet business-driven deadlines. Unexpected longevity of software systems causes tech debt to accumulate to a point of no return despite well-intentioned engineers. This is a scenario where technical renovation is driven by organically outgrowing your existing systems.

These are some nails for the hammer that is Technical Renovation. Many companies find it difficult to prioritize technical renovation due to competing business priorities, the fear of operational disruptions, and the substantial costs associated with large-scale migrations. Here are some battle tested strategies to adopt for technical renovation projects to mitigate risks of disruption and increase the likelihood of success.

Strategies for Technical Renovation

Evolutionary Architecture

The traditional approach to software architecture design is rigid and involves pre-planning. However, modern complex systems cannot be fully designed in advance. In a rapidly changing environment, software development faces numerous uncertainties, necessitating a flexible approach to system architecture. Evolutionary architecture recognizes that software systems must evolve to meet changing business requirements, technological advancements, user needs, and organizational constraints. Here are some key aspects of an evolutionary approach to software architecture:

• Identify fitness functions – Begin by clearly defining the business objectives and the quality attributes that are critical to your system. These could include performance, scalability, security, maintainability, and usability. For each quality attribute, establish measurable criteria that can be used to evaluate the system’s health. For example, if performance is a key attribute, you might define a fitness function that measures response time or throughput under certain conditions. This is a good exercise to reveal conflicting priorities and recognize that optimizing for one quality attribute may impact others. For instance, enhancing security might affect performance. Use fitness functions to find the right balance that aligns with the overall business priorities.

• Continuous delivery and evaluation – Continuous Delivery ensures that software can be reliably released at any time while continuous evaluation involves monitoring the software in production to gather feedback on its performance, reliability, and user experience. This enables teams to deliver high-quality software rapidly and reliably. It encourages a culture of continuous improvement where feedback from fitness function evaluations is used to drive future architectural decisions.

• Lean into Incremental changes – Smaller, incremental changes are easier to measure and deploy, reducing the likelihood of introducing significant regressions. Continuous delivery and evaluation are prerequisites to supporting an iterative approach to software development, where feedback from users and monitoring tools is used to make informed decisions about future development. This iterative cycle allows teams to respond quickly to changing requirements and constraints.

Make It Work, Make It Right, Make It Fast

While Kent Beck’s quote is commonly applied to writing code, it also applies to software engineering more broadly, including renovation initiatives.

• Make it work – This initial phase focuses on gaining confidence that your problem can be solved, one way or the other. From the standpoint of writing code, it is perfectly acceptable to take shortcuts like hard-coded inputs in this phase and allow yourself to write code that may not be pretty or easy to read in the spirit of proving feasibility. In the context of renovation, use this time to validate your assumptions and technology choices by addressing common use cases; and eliminate ineffective solutions along the way. A successful outcome of this phase is a Proof of Concept.
• Make it right – Once you’ve confirmed the viability of your solution, it’s time to “Make it right”. From a coding perspective, this means focusing on improving readability, refactoring your code, and strengthening your test cases. For your renovation project, now is the ideal time to ensure edge cases are covered and your fitness functions are satisfied. Now is also a good time to test against a small cohort of real users to see how your solution holds up. An MVP – a minimum viable product that is a natural extension of the proof of concept is a good outcome of this phase.
• Make it fast – From a coding perspective, this phase often focuses on enhancing performance like optimizing your code, implementing caching, etc. For renovation initiatives, “make it fast” encompasses more than just performance improvements. It encompasses optimizing the speed of execution by improving documentation, setting up continuous delivery, etc; and improving operational efficiency by establishing monitoring and observability practices. These efforts accelerate your software development process, aligning perfectly with the “make it fast” objective. The outcome of this phase is production-grade software that is ready for prime time.

Here is a visualization of how each of the phases lines up for a renovation initiative. This structured approach to tackling the different aspects of a technical renovation helps break up a daunting endeavor into manageable milestones.

Deprecation-Driven Development

Deprecation-driven development is like spring cleaning, where the focus is on what we gain by clearing out the old rather than what we lose. Just as tidying up your home involves removing clutter to create a healthier living space, systematically eliminating obsolete technology is essential for maintaining healthy software systems. It is important to weigh the tradeoffs before renovating a system and be honest when the return on investment doesn’t justify the renovation effort. Not all features are equally important for the business, when you identify a legacy feature that isn’t critical to your organization’s growth or success, consider scoping it out of your renovation initiative. Better yet, deprecate it entirely! Maintaining and operating software is often the most resource-intensive part of the software development lifecycle, especially for legacy systems. Therefore, I argue that investing in deprecating features is an important step in your renovation journey.

A good example of deprecation-driven development is Netflix discontinuing its DVD service. The key consideration was the balance between the cost of running the DVD service and how much it contributed to the business. As the number of DVD subscribers steadily declined, it became increasingly challenging to justify the cost required to deliver a top-notch service experience for Netflix’s DVD users. Once the decision to deprecate the DVD service was made, the focus shifted away from investing in DVD-related technology, allowing resources to be redirected towards more impactful areas of the business that align with the company’s long-term vision. This example underscores the significance of making tough decisions to phase out outdated offerings in favor of pursuing more promising business opportunities, ensuring the company’s continued growth and relevance in a rapidly evolving entertainment ecosystem.

Intentional Organization Design

As businesses evolve, so should their organizational structure. An effective structure facilitates seamless communication and collaboration across the organization, beyond that, organizational design also impacts their system design. Conway’s Law highlights the connection between team organization and system architecture, suggesting that the structure of teams influences the architecture of the systems they develop. Therefore, for organizations embarking on a technical renovation, it is wise to first step back and evaluate their current structure. This assessment can help identify beneficial changes to streamline communication and reduce the need for cross-team coordination, this is crucial for both innovation and execution velocity. As an example, let’s review two common ways of structuring an engineering organization

• Functional Teams – groups engineers by function, such as backend, web developers, and mobile developers. This design allows engineers to quickly ramp up and become experts in their specific functional areas. Each product feature likely requires coordination across multiple teams as changes often span different layers of the stack.

• Cross-functional Teams – organizes engineers into full-stack teams based on common product deliverables. This design minimizes cross-team coordination for features owned within a vertical, as each team can handle all aspects of a product feature. This structure scatters technical experts across different teams, requiring additional coordination for the specialists to align on best practices, share learnings and cross-pollinating ideas relevant for their areas of expertise.

Each has pros and cons, and the right choice depends on whether deep functional expertise or seamless collaboration within a product vertical is the priority. In conclusion, it’s essential to strengthen the communication paths that are most critical for your organization and its architecture, as not every communication path can be equally robust.

Conclusion

The renovation strategies I discuss above may seem ambitious, but they are intentionally aspirational. There are no one-size-fits-all solutions for technical renovations; the ideal approach is highly context-dependent. Your strategy and decisions should be debated on a case-by-case basis and account for your organization’s unique circumstances and goals.

Start with the right perspective, the perspective that legacy systems are inevitable in successful companies because they are a byproduct of success. For continued success, it is crucial to use the right tools for the problem you’re facing, while refactoring can address certain issues, a technical renovation might be necessary for others. Invest in technical renovation when it is justified, and when you do, focus on building systems that are evolvable to keep pace with your business needs. Break down the daunting task of renovation into manageable milestones: make it work, make it right, and finally, make it fast. Exclude legacy features from your renovation plan if the return on investment is not justified.

Regardless of your company’s growth trajectory, size, or operating margins, transforming legacy systems will present new challenges. So embrace a growth mindset, seek feedback, and learn from your mistakes to make the most of your renovation journey.