Why people find it hard to review code

Code reviews are often used as part of the software development process to control which code changes are allowed to make it into the code base. Empirical research consistently shows that code reviews have a positive impact on software quality. However, there is also evidence that code review requires a lot of effort, might not always be effective, can induce confusion, and can harbour toxic conversations and bullying.

In this week’s paper summary, we look at how developers review code and their (negative) experiences with tools in the code review process. These findings are based on a series of interviews and surveys that involved software developers from two multinational companies.

The researchers identified two main themes centred on misalignment in the code review process and its tools, which they call tool-task misalignment and process-task misalignment.

Tool-task misalignment occurs when a code review tool does not meet the needs of a reviewer when they are conducting a code review. The interviews revealed three causes for this type of misalignment.

Code review tools typically make it possible to comment on changes that are part of the diff, but are unable to facilitate the full range of conversations that are necessary for a proper review. For example, a reviewer might want to comment on a piece of code that is affected by the change but not part of it, while a front-end developer may want to show the effect of a change using videos and images.

Several participants mentioned that they could not find the information they needed for their review. The effect of changes or even the changes themselves may be hard to understand, and thus require that reviewers manually pull the changes into their editing environment which offers more visual and contextual support. Moreover, different reviewers have different “units of attention”, i.e. they want to focus on different things. Some reviewers are interested in all the details, while others only want to understand the impact of a change.

Finally, the size of a change may create obstacles. Large changes are harder to review; not only due to increased complexity, but also because review tools may become impractical and slow.

Process-task misalignment is about communication among team members and other stakeholders, and the way reviews are organised.

For instance, developers described a team practice where reviews are everyone’s responsibility, but in practice, this is far from the case: the review load is not shared evenly, and developers self-select review tasks. Furthermore, it is often unclear who is responsible for following up on comments in a review. In some cases it’s up to the author, while others believe that the reviewer is responsible for making sure that comments are processed.

There is also frustration from both authors and reviewers about the completion of reviews. It can take a while before a change is reviewed, especially if the change is large. Although some developers try their best to review changes within a reasonable time so that they don’t block colleagues, many still need to be reminded to review their colleagues’ work via side channels (side note: Like Slack and Teams).

Automation of code style checks is meant to reduce the cost (in terms of effort and frustration) of code reviews. However, the suggestions in these automated review comments do not always align with the preferred coding style of individual authors. Some even believe that authors should have the freedom to write code in their own style.

Finally, some developers worry about the experience of using a code review tool. New developers, especially those that have insecurities, might not be comfortable with feedback in a relatively public space.

Developers spend about 6 hours per week on code reviews, with a median of 1 hour per day for an average workday.

When asked about the perceived benefits of code reviews, the effect on software quality was mentioned most often, followed by the potential for knowledge sharing:

• Improved overall software quality

• Improved knowledge sharing

• Improved collaboration

• Improved ability to integrate new or less experienced team members

• Improved adherence to common standards and guidelines

• Reduces overall project costs

• Improved customer satisfaction

• Reduced time to market

Review obstacles and challenges are common. Respondents experienced tool-task misalignments in about every fourth code review. When asked about misalignments in general, the top three were related to (lack of) completion of reviews, inability of review tools to provide all necessary information, and the size of proposed changes.

The survey also uncovered a few other obstacles with reviews that are related to the reviewing experience, like a lack of interest in the code under review, code review being boring, and discomfort giving critique.