Are developers aware of the architectural impact of their changes? (2017)

Harvard Business School’s Baker Library / Bloomberg Center gets a new European post-war addition

Architecturally clean systems are easier to maintain. Changes to a system therefore shouldn’t degrade its architecture. Paixao, Krinke, Han, Ragkhitwetsagul, and Harman studied four large projects to better understand whether and how developers take the system’s architecture into account when making changes.

Why it matters

Architectural decisions strongly impact a system’s maintainability.

For example, studies have shown that coupling and cohesion clearly affect various aspects of software quality, including maintainability, comprehensibility, code smells, and bug-proneness.

A better understanding of how developers deal with architectural changes is necessary in order to design proper tools and processes that prevent developers from making changes to a system that degrade its architecture.

Cohesion and coupling

Cohesion is basically the number of different things a module does. Ideally, a module solves only one problem. In other words: software modules should have high cohesion.

Coupling is the extent to which modules depend on each other. Modules that strongly depend on each other are hard to evolve separately. Therefore one should strive for low coupling.

How the study was conducted

The authors study changes from four Java projects with a large number of merged reviews. For each project, the authors extract all reviews and commits that represent the state of the system immediately before and after each reviewed merge.

The structural cohesion and coupling are computed on the module dependency graphs of the “before” and “after” states. This is done twice for each state: once for the overall architecture and once for a single package. The resulting numbers make it possible to determine the architectural impact of each merged change.

What discoveries were made

Most changes that significantly impacted the system’s architecture were made to introduce a new feature, enhancement of an existing feature, refactoring, and bug fixingThese are typically cases in which the wrong thing was built and significant changes are necessary. Together, they account for 90% of all significant architectural changes.

Discussions about the impact that a change will have on the system’s architecture are conspicuously absent in most reviews, especially if that impact is negative. This suggests a lack of architectural awareness among developersThis claim might sound a little bit far-fetched at first, but it’s not that outlandish when you think about it: all projects in this study are open source projects, with distributed development. This makes it likely that any discussions about architecture would have taken place in code reviews, rather than in email threads, online chat, or in-person..

Reviews of changes that have refactoring as their primary goal generally do include discussions about architectural impact. Coincidentally, the impact tends to be more positive in these cases.

The impact is not always positive however. Sometimes refactoring results in a (partial) architectural deterioration. Reasons vary: in one case it’s because the developer did not realise the full ramifications of their changes, in another it’s because developers also appear to consider aspects other than coupling and cohesion.

In any case, it appears that one should strive for increased awareness of the architectural impact of changes among developers.

The important bits

  1. Changes of which the architecture is discussed during review tend to have higher structural improvement
  2. The architectural impact of a change is discussed in only 38% of reviews, which suggests a lack of awareness among developers
  3. Tool support might help increase architectural awareness and slow down or even prevent architectural degradation