Tackling gender biases in AI-driven systems

Gender is not just about biological characteristics, but also cultural norms, social roles, and expectations. Some of these give rise to gender bias: the unequal treatment or representation of individuals based on gender. Such bias can lead to stereotyping and discrimination, and cause significant harm to individuals and society.

Gender bias can be especially problematic in software design and development. Because software is cheap to distribute, a biased system can have a widespread impact once it’s been developed and deployed “in the wild”.

Over the past few years, AI has become a major part of both software development and the software products themselves. Given the over-representation of men in tech and the data on which AI is trained, AI might quietly undo decades of advances in gender equality, resulting in software that perpetuates gender ideologies that disadvantage women.

Well-known examples include the names and default voices of popular virtual assistants, such as Alexa, Siri, and Google Assistant, all of which use a female voice because they're perceived as "supportive" and "humble". But this also reinforces the stereotype of female servants and secretaries.

This week’s paper looks at how design can help prevent gender bias in AI-driven systems. The authors reviewed guideline documents from various EU-based institutions and arrived at four recommendations for designing more effective guidelines to tackle gender biases in AI.

The researchers analysed ten European guideline documents for the presence of several aspects they argue should be included, such as the identification of gender bias as a specific issue, proposed solutions to prevent gender bias or discrimination, and proof that those solutions work.

The analysis shows that in practice guidelines often cover only some aspects, and none address all of them. For example, only one of the reviewed documents considers gender bias on its own. Most guidelines take a broader approach and try to address multiple biases at once. This raises some concerns, as it may result in underdeveloped solutions that do not work well.

All ten guidelines propose at least one solution to mitigate bias and discrimination, but only three present an approach to implement the solution in practice. The depth and specificity of these plans vary. For example, Utrecht University’s DEDA is a generic toolkit that can be applied to any business or organisation (with some adjustments), while Telefónica (side note: A private company in Spain) uses a custom-made approach specifically tailored to its own activities.

In practice, proposed solutions are most commonly some form of checklist or questionnaire, followed by the creation of diversity teams or ethics committees, training courses, and the implementation of bias prevention algorithms (side note: Without ever specifying what is exactly meant by this, so I’m going to assume that some of these documents are mainly for marketing purposes.). Several documents call for audits, certifications, or the creation of data custodian departments to control the type of training data provided to engineers.

Then the third aspect I mentioned: do the proposed solutions actually work? Only four of the documents state that their solutions were “successful”. Taking Telefónica as an example again, their approach was adapted from methods already proven effective elsewhere, and they have also published several papers on the matter. In the case of other private companies, it is not clear whether internal stakeholders were consulted on proposed solutions. This is a potential pitfall, as stakeholder participation at every stage is crucial for keeping them motivated and ensuring gender bias is properly taken into account. Interestingly, none of the four documents present quantitative evidence that their proposed solutions work.

A key insight from the analysis is that guidelines should be structured along two dimensions: general principles for AI ethics and specific issues falling under those principles. Governments should likely define overarching principles like fairness and non-discrimination, while companies should outline their approaches to specific issues within those broader principles.

Newly written guidelines can be made better by following these four recommendations:

1. Proposed solutions should be specific to a single issue and must be more than a simple checklist. For gender bias specifically, this is a bit of a problem, as there are not that many design tools dedicated to addressing it.

2. Everyone within the organisation should be well-informed about the issues related to AI ethics and understand how solutions are intended to address those issues, regardless of their current level of understanding of AI and its ethical implications.

3. Any proposed solution should be tested with stakeholders (in advance!) and allow for feedback. The success of solutions should also be quantified and shared with relevant stakeholders to ensure accountability and trust.

4. Guidelines issued by governments should include a list of existing initiatives that have already proven successful to serve as a reference point for companies seeking to implement ethical AI practices.