Everything that management needs to know about technical debt

Companies that develop software generally make their money by shipping as many features as quickly as possible. In an ideal world, those new features would be built in a clean way that follows sensible design principles and does not erode the system’s architecture. In practice developers are often forced to take shortcuts so that new features can be delivered to customers more quickly – at the cost of reduced code quality.

This reduction in code quality is often referred to as technical debt; the idea being that it will – or at the very least should – be repaid at some point in the future. As long as the codebase has an outstanding debt, it requires its developers to pay “interest” in the form of confusion, frustration, unexpected malfunctions, and possibly other “-ions”.

Unfortunately technical debt rarely receives the attention it deserves. The benefits of paying down technical debt are not always clear to managers, who have to weigh vague long-term benefits that improved code quality reportedly provides against the clear short-term benefits when new features are shipped.

This week’s paper shows the direct impact that technical debt (in the form of reduced code quality) has on several aspects of software development, and its possible consequences for the bottom line.

The study is based on two major sources of data: technical debt, as measured by the proprietary CodeScene tool, and everyone’s favourite project management tool, Jira, which keeps track of the number of defects and how much time it takes to resolve them.

CodeScene claims that it can compute the code quality in any given codebase using their proprietary Code Health metric. I won’t describe this metric in detail here. Primarily, because for our purposes it doesn’t really matter how it works, but also because very few tools are capable of reliably measuring technical debt. What’s important, is that CodeScene uses the Code Health metric to classify each module in a codebase as either Healthy, Warning, or Alert (side note: Based on seemingly arbitrary cut-off points) and that it reportedly works pretty well.

These health ratings can be combined with information from Jira. By looking at the frequency and resolution time of defect issues one can determine whether a relationship exists between code quality and business outcomes.

Somehow, the researchers have managed to convince quite a number of CodeScene’s customers to participate in this study: they collected data from 39 proprietary codebases (side note: This is something you don’t see very often…), along with data from their Jira instances. Due to confidentiality agreements not much can be disclosed about the analysed projects, but the authors assure us that it’s a fairly diverse sample with codebases that are from different domains and use different programming languages.

The results of the analysis on these 39 codebases are fairly straightforward.

Firstly, there is a statistically significant difference in the number of Jira defects between the three Code Health categories (Healthy, Warning, and Alert). This suggests that code quality has a significant impact on the number of defects in a file. On average, Alert code results in 15 times more reported Jira defects than Healthy code.

Secondly, defects in low-quality code also take longer to fix. The researchers measured the time between the time a developer first moves a bug issue to the “In progress” column and the time of the last commit for that bug issue, and compared this to the code quality. On average, development takes 78% longer for code in the Warning category and 124% longer for code in the Alert category.

Issues in low-quality code don’t just take longer to resolve: the development time for such issues is also much more unpredictable. The maximum development time for code in the Alert category is a whopping 9 times longer than the maximum development time for code in the Healthy category!

These findings show how important it is to actively manage technical debt. Neglecting technical debt results in a higher number of defects, which also take longer to resolve. Mre resources need to be spent on unplanned work, which leads to cost overruns. Moreover, less time remains for implementing new features that keep the project competitive (or profitable).