Empirical study on the relationship between developer’s working habits and efficiency

There’s plenty of evidence that suggests that working long or late hours is neither good for you nor your productivity, but that doesn’t stop most developers from doing it anyway. Rodriguez, Tanaka, and Kamei analysed the actual working habits of software developers and the effect that these habits have on their efficiency.

This is where I normally explain why I picked this paper by highlighting a particular problem or raising some question that has never been answered before (side note: In a peer-reviewed work, of course.).

This week’s paper is a bit different: it was originally submitted for the MSR 2018 Mining Challenge, which aims to bring researchers and practitioners together by letting them analyse a common dataset.

In this case that dataset is a set of repository and IDE interaction data that’s gathered from FeedBaG, which according to the challenge’s website is

a general-purpose interaction tracker for Visual Studio that is built as a plugin to JetBrains ReSharper framework

Don’t worry if that sounds a bit vague: all you need to know is that it contains data on the actions that were performed by 81 developers within their IDEs.

These actions include:

• Any activity that occurs within the IDE, e.g. mouse clicks, pointer movements, and key presses. Any activity that occurs within five minutes of another activity, is considered to be part of the same programming session. This allows one to estimate (side note: Programming often also includes activities that do not happen within the confines of an IDE, like whiteboarding and documentation. Those data are not included in the dataset, so this as close as you can possibly get.) how much time someone has spent programming.

• Initiation of builds and test runs. The proportion of successful builds and test runs can be used to calculate success and failure rates, which allow one to infer the efficiency of a programming session.

The authors use the timestamps of these two variables to determine whether certain working habits truly impact efficiency.

The authors look at the data from three different perspectives.

When grouping data by day of the week, it’s clear that:

• Developers work twice as many hours during business days as during the weekend;

• Programming sessions last longer on Saturday than on other days of the week, which suggests a lower number of interruptions;

• Saturday has high success rates and low failure rates. It’s easy to attribute this to the length of programming sessions (side note: “Being in the flow/zone”) – however, Wednesday shows similar rates without long programming sessions.

Grouping data by hour of the day yields the following observations:

• Overall, programming happens throughout the day, but most development occurs between 9:00 and 0:00;

• Total working time peaks between 15:00 and 17:00, presumably because many developers try to finish their work before the end of their working day. Coincidentally this is also when the failure rate peaks, possibly due to lack of time or other stress factors;

• Success rates are significantly higher in the evenings (18:00–0:00).

Finally, grouping data by length of programming sessions shows that:

• Success rates are fairly consistently around 90% at first, but start to drop to 80-ish% when a developer works for two straight hours without any breaks;

• Failure rates on the other hand gradually increase from about 20% to 30% after two hours of continuous work.