Chuniversiteit.nl
The Toilet Paper

Empirical study on the relationship between developer’s working habits and efficiency

Bottom line: don’t overdo it.

A tired developer that doesn’t produce a lot of working code
Is that solution a good idea? You may want to sleep on it.

There’s plenty of evidence that suggests that working long or late hours is neither good for you nor your productivity, but that doesn’t stop most developers from doing it anyway. Rodriguez, Tanaka, and Kamei analysed the actual working habits of software developers and the effect that these habits have on their efficiency.

Why it matters

This is where I normally explain why I picked this paper by highlighting a particular problem or raising some question .

This week’s paper is a bit different: it was originally submitted for the MSR 2018 Mining Challenge, which aims to bring researchers and practitioners together by letting them analyse a common dataset.

How the study was conducted

In this case that dataset is a set of repository and IDE interaction data that’s gathered from FeedBaG, which according to the challenge’s website is

a general-purpose interaction tracker for Visual Studio that is built as a plugin to JetBrains ReSharper framework

Don’t worry if that sounds a bit vague: all you need to know is that it contains data on the actions that were performed by 81 developers within their IDEs.

These actions include:

  • Any activity that occurs within the IDE, e.g. mouse clicks, pointer movements, and key presses. Any activity that occurs within five minutes of another activity, is considered to be part of the same programming session. This allows one to how much time someone has spent programming.

  • Initiation of builds and test runs. The proportion of successful builds and test runs can be used to calculate success and failure rates, which allow one to infer the efficiency of a programming session.

The authors use the timestamps of these two variables to determine whether certain working habits truly impact efficiency.

What discoveries were made

The authors look at the data from three different perspectives.

Day of the week

When grouping data by day of the week, it’s clear that:

  • Developers work twice as many hours during business days as during the weekend;

  • Programming sessions last longer on Saturday than on other days of the week, which suggests a lower number of interruptions;

  • Saturday has high success rates and low failure rates. It’s easy to attribute this to the – however, Wednesday shows similar rates without long programming sessions.

Time of the day

Grouping data by hour of the day yields the following observations:

  • Overall, programming happens throughout the day, but most development occurs between 9:00 and 0:00;

  • Total working time peaks between 15:00 and 17:00, presumably because many developers try to finish their work before the end of their working day. Coincidentally this is also when the failure rate peaks, possibly due to lack of time or other stress factors;

  • Success rates are significantly higher in the evenings (18:00–0:00).

Continuous working time

Finally, grouping data by length of programming sessions shows that:

  • Success rates are fairly consistently around 90% at first, but start to drop to 80-ish% when a developer works for two straight hours without any breaks;

  • Failure rates on the other hand gradually increase from about 20% to 30% after two hours of continuous work.

Summary

  1. Long, uninterrupted programming sessions are more likely to happen on Saturdays

  2. Wednesday and Saturday seem to be the most efficient days for programming work

  3. Builds and/or tests runs tend to fail more often right before the end of working days, which is typically between 15:00 and 17:00

  4. Programming continuously for more than two hours correlates with higher build and/or test failure rates