Chuniversiteit.nl
The Toilet Paper

Expressions of sentiments during code reviews: Male vs. female

Men and women express their feelings in code reviews differently. That’s not necessarily a good thing.

Excerpt from American Chopper meme
This might not be the best way to get your point across

This study looks at the emotions that men and women express in their review comments, how they differ, and possible consequences for gender diversity in software engineering.

Why it matters

Until the 1960s programming was largely done by women. Since then, the field has become increasingly male-dominated through professional associations, hiring campaigns that discouraged women, and personality tests that were biased against female applicants.

The few female software developers that remain in the field today report that they experience sexual harassment, get fewer opportunities, and .

But it’s not just about getting pull requests to be approved: the language used in the reviews matters as well.

Some believe that review comments – like other forms of feedback – should be seen as a gift. It’s easy to see how this can be true if a comment contains encouraging or heartwarming words. It gets a bit harder when it includes criticism, snark, or is full of expletives. (How) do comments between males and females differ?

How the study was conducted

The code reviews of six popular open-source (C, C++ and/or Java) software projects were mined for review comments.

Due to practical reasons, the authors only focus on 3,570 so-called non-casual developers, i.e. people who have contributed at least five changes to a project.

The authors used the genderComputer tool to guess developers’ genders. Since most developers are male, the authors then manually validated the gender of all 937 (supposedly) non-male developers by looking up their profile pictures in the review tool or on social media.

The SentiSE sentiment analysis tool is then used along with SentiSense, a lexical sentiment database, to determine the sentiment of each developers’ review comments.

What discoveries were made

More than 85% of review comments do not express any sentiments. Of the comments that do, most have a negative tone. Comments that are written by women are less likely to express any positive or negative sentiments. However, women do express sentiments more often when they interact with other women.

Men seem to express their sentiments more consistently, although in three of the projects, men write considerably more negative comments for their female colleagues, while withholding positive comments. It is likely one of the many reasons why so many women leave the computing field after a few years.

When we look at the type of sentiment words that are used, we can see that men are more likely to use words that are related to strong emotions, like surprise, sadness, love, joy, hate, disgust, fear, and anger. In three of the projects, comments written by women seem to show more anger and digust, .

Some comments contain emoticons. Positive emoticons are used more often than negative ones, and mostly by men. This is somewhat surprising, as many studies show that women generally use more emoticons than men. The authors suspect that this may be due to a lack of diversity: because women are often misinterpreted by men and perceived as emotional, they are taught that it’s best to show as little emotion as possible.

Review comments rarely contain expletives or swear words, which is not very surprising considering that . Male developers appear to swear more often than female developers. Words like “crap”, “damn”, and “screw” are commonly used by both genders. Expletives that are highly offensive (especially towards females), like “bitch”, “bastard”, “fuck”, and “jerk” were only used by males.

Summary

  1. Male developers express sentiments in review comments more often and using words with stronger sentiments than women

  2. Women are more likely to express sentiments to other women than to men, while men are primarily harsher against women

  3. Female developers are less likely to use emoticons in review comments

  4. Male developers swear a lot more than female developers and tend to use more offensive words as well