Every product evolves in its own mysterious way. Even the experienced team cannot predict every twist in the product life cycle. Simple features could be transferred into monstrous multi-condition workflows. Some fast-developed side scenarios could become the most used ones. Even the popularity of the product could drive unexpected performance issues. And it’s completely normal to adapt software for market needs. The only way to have a reliable and predictable product is to reserve some time for fixing technical debt and providing a proper refactoring process.
It’s better to include some definitions of terms in this article. Technical debt, or tech debt, is the accumulated compromises in the codebase, that may occur during fast development or other fluctuations. The refactoring process is more about improving the product in general. It may involve activities related to performance, better code structure, and simplicity of the solution. However, sometimes these two concepts can be very close. For example, a few features full of tech debt may require proper refactoring of the whole module to fix the problem once and for all with new ideas and reimplementation.
And here’s the trick. In general, good software developers have plenty of ideas for product improvements. “Okay, we could tweak some stuff during this feature.” “Oh, that’s not so important but it would be nice to fix this thing in this side module”. Still, it is crucial to allow your team members to have an opportunity to enhance the codebase, as it helps build real involved teams. However, in this article, I would like to reveal the refactoring process from a manager’s perspective. The examples above have one problem – they might be opaque for the whole team except devs. While QA and managers have their plan to release proper and stable software on schedule, some hidden improvements can create unexpected twists and nervous retests during the release. Or even new bugs in production if through the regress stage, some undocumented changes go unnoticed. So, code improvements are necessary, but they should be manageable.
How to build a manageable refactoring process in your team
The solution is complex. Agile is our best friend because common principles and rituals of Agile development methodologies cover problem spots. You need to:
- Build technical debt backlog and allocate resources regularly.
- Establish proper grooming and planning processes to discover potential improvements.
- Declare processes in case of unexpected problems during the iteration.
By grooming, I mean communication between team members before iteration aimed at defining the iteration scope, specifying requirements, decomposing tasks, estimating efforts and prioritising future work items. The planning process is connected with the resulting scope of iteration. Not every groomed task may be included in iteration.
Let’s dive into each topic.
Building technical debt backlog and resource allocation rule
The easiest way for every developer to build a tech debt backlog is to use “To do” tags in the source code. , you could search the codebase, improve something, and send pull requests for approval. But that is not enough. It doesn’t help team members to provide proper planning and to be sure about the release scope.
The better alternative to using “To Do” is to establish a process for creating the tasks for future changes. If the developer finds some potential trouble spot or has an idea for improving the codebase, they should create a work item in the ticket system (Jira, Azure DevOps, or whatever system the team uses). It would be overkill to require engineers to provide a detailed description of their idea for every team member, but the explanation should be clear enough for the team lead to understand the key point and scope of changes. This covers the first step – how we can create a list of potential tasks and provide a high-level description of future changes.
Step two is to make it understandable for everyone. This task could be handled by the dev team lead, release manager or product manager, depending on their qualification. The outcome should provide the following details:
- What are we doing? – A high-level explanation of the idea.
- Why we are doing this? – A description of how these changes could improve development speed, reduce the risk of instability caused by spaghetti code, or increase software performance.
- How important are these changes for future development? – The priority of changes. For example, without these changes, the cost of future improvements or business features may grow exponentially.
- What risks do these changes create? – A list of affected features or modules to help QA verify the changes.
If the work item has answers to all these questions, it helps mitigate potential problems in the future. However, having a backlog alone is not sufficient for manageable refactoring. You need to plan possible changes, and they should be a constant part of your process. Sure, you will face the pressure of business features and market demands, where the temptation to omit tech tasks is high. But without proper maintenance, you are going to face even bigger problems in the future.
Recommendation for the tech backlog process is:
-
In every iteration, the team must reserve time for tech improvements.
-
If there’s any idea for improvement, it should be formalised as a work item with a proper description.
-
Work items should participate in the grooming process and be discussed by the whole team until all benefits and risks are identified and estimations from all team members are gathered.
-
Work Items shall be planned into Agile iterations according to their estimates.
The team should be acknowledged of possible tech debt volume in the iteration. Reserved time could be increased if necessary, but should never be less than the regular amount. This helps to encourage engineers to create new tasks in the backlog and prioritise them. Everyone knows that their suggestions could be implemented and the backlog is going to shrink someday, not grow into an enormous demoralising list.
Using grooming and planning processes for potential tech debt disclosure
Sometimes, tech debt tasks may be revealed during discussion and estimation, while the team faced unexpected obstacles that made realisation less reliable and flexible. For example, you may realise that there are similar features and creating a brand-new one would only aggravate the codebase. But those features do not contain the business functionality required in new ones. And it’s better to create a unified service that connects all similar features to simplify maintenance in the future. Still, this change may affect and destabilise old features and that’s where the challenge lies. Maybe there’s a way to develop new features cheaply and turn a blind eye to imperfections. Or maybe right now is the finest opportunity to create a unified service while there are only a few similar features. The most important thing is to establish a process that helps team members make a decision based on revealed facts and properly groomed estimations. It’s crucial to have a system that prevents situations where such decisions should be adopted during the implementation phase in a committed backlog.
If your iteration stages work well, the disclosure of such moments will happen automatically via a proper grooming and planning process. There are a few basic rules and steps that could protect the team from unexpected obstacles in most cases:
- The grooming backlog should contain tasks with clear and feasible requirements.
- The grooming backlog should be shared with team members before discussion and estimation.
- All team members must be prepared and familiar with the tasks before grooming.
- If any backlog item requires additional investigation or reimplementation, the concern should be discussed.
- Requirements for problematic backlog items should be finalised based on revealed peculiarities. All potential problem areas should be documented.
- Each backlog item should be estimated.
Problematic backlog items could be decomposed into separate tasks and planned according to priorities. Decisions about the implementation strategy may be up to the release manager, according to risks and deadlines. But this approach helps to document all decisions. Even if the tech debt task was postponed, it is still added to the backlog. , these tasks should be reviewed and scheduled. This process fixes scenarios where some good and necessary ideas are forgotten during a hectic iteration.
Establishing process in case of unexpected problems during the iteration
Still, even with a well-maintained tech debt backlog and a working grooming and planning process, it is possible to run into unexpected and time-consuming obstacles under development. Software products could be complicated, especially old or containing rich functionality.
Using Agile practices helps gather information early and make a proper decision. One of the easiest solutions is daily meetings.
If an engineer faces some problem, it should be popped up during the meeting and discussed later. Every obstacle is unique and may consume different amounts of time. It doesn’t matter whether the change will be implemented in the current iteration, instead of a ‘Nice to have’ feature, or requires a complete revision of the iteration scope. The issue should be created as a typical tech debt task in the tracking system, decomposed and described as any other similar task in previous articles. Consistency is the key, and the team have to know how to handle these situations.
Criticism of additional bureaucracy and how to answer to it
All these methods may seem like an additional way to take the whole team’s time with useless bureaucracy. And it might be so if only the absence of strict and clear rules didn’t create hard times for everybody. It’s OK not to follow these recommendations in short-term projects or at the minimal valuable product (MVP) stage. However, working in production with quality responsibilities and big products requires a well-defined internal process system. Let’s go through the most common objections.
Creating such tasks is time-consuming and sometimes it’s faster to make changes in code than to describe them
And that’s true. Describing your tasks in natural language can sometimes be harder than fixing the code. But here are some solutions:
- Creating a system of templates in the ticket system can be helpful. It allows tasks to be added quickly, and filled with the correct parameters and links.
- It’s great to have some ticket policies on the corporate wiki, such as Confluence / Notion / SharePoint or any other team documentation platform. These pages can provide good examples of correctly created tasks. It’s in the team lead’s best interest to maintain technical documentation and polish templates to help any other team member submit clear and understandable tickets.
- At the basic level, it’s enough to oblige engineers to create tasks with high-level descriptions without extensive articles on strategic vision or the potential of their suggestions. The description should help the reviewer (usually the team lead) to get the main idea of changes. , the team lead can validate the suggestion and fill it with additional details according to the process. This also helps to understand if these changes are even necessary and provides some quality filter.
- If the developer has time to implement the suggestion, the feature-branch policy can be very helpful. Still, it’s necessary to create a task, because it’s useful to have a rule for the creation of feature branches named by ticket. But as a result, we get a satisfied engineer, who has implemented some piece of tech debt, ticket and linked implementation that can be groomed and planned for verification in a suitable iteration, without the risk of unexpected destabilisation.
All these changes are highly technical, and the description doesn’t help because nobody will get the idea
Maybe. But it depends on the explanation. The priority is the description of what could go wrong and what functionality might be destabilised. However, it’s useful to write about the impact of change because it helps identify additional possibilities and scenarios. Also, even the most deeply technical ideas could be described in simple sentences using natural language. If they cannot, there may be something wrong with the idea itself, and that could be a potential risk, the whole proposal should be reconsidered.
Just write something like:
“Method “XXX” consumes too much RAM on every call. Creating an additional cache for this method helps reduce RAM consumption and speed up all APIs. The method uses data that changes rarely, it’s sufficient to drop the cache on restart. The changes are safe, but may potentially affect the following features: XXX, XXX, XXX…”.
In some cases, this would be enough. In others, this suggestion may trigger the discussion, because the engineer might forget about some old but still used functionality where this cache could cause issues. During the grooming process, the idea will be revised, and the team will find a compromise.
All these policies only prevent users from receiving new enhancements
Stability and reliability are better than a few percentage improvements in some feature execution time. Nobody will be disappointed about missing a potential performance boost, but it’s easy to tarnish a product’s reputation.
Do we need all this bureaucracy for a simple code style that will never do any harm in 99.99% of cases?
The idea is to streamline processes and help assess risks. Engineers should have the opportunity to maintain the codebase and provide some improvements. For things that are small and don’t destabilise the whole product, it’s possible to create aggregative work items that can be completed during iteration. These tasks still need to be reviewed as pull requests but it isn’t necessary to formally announce them to the team.
Conclusion
Constant product improvements are critical for business and help preserve overall quality. If you keep the technical debt and new ideas on the shelf, you will get stuck with an outdated product and soon struggle to find expert engineers to maintain it.
On the other hand, these tasks compete with business features and other ideas that can drive business to new horizons. These recommendations about technical tasks backlog can help reveal and assess the importance of these tasks not only for team members but also for stakeholders. They help present these ideas in natural language and keep and protect time for implementation. It burdens engineers with additional actions, but in the end, it helps the whole team to deliver high-quality and reliable products. And the responsibility of a manager is to choose the right instrument or policy to maintain this process.
