Agile teams don’t use a lot of data in their retrospectives

Many agile teams hold a retrospective meeting at the end of each sprint, in which team members reflect on their recent work by identifying what went well and what can be improved, and planning actionable changes to achieve those improvements. When done well, retrospectives can improve a team’s workflow and productivity, and build trust and morale among team members.

Much of the time in retrospectives is spent on gathering data. However, most of that data is subjective data such as anecdotal feedback or instinctive judgements, which is somewhat curious in a field that is otherwise increasingly data-driven and attempts to shove telemetry into everything. Incorporating objective data (side note: e.g. test results, number of bugs or deployments) into retrospectives may reduce memory biases by providing a factual record of events, and allow teams to challenge assumptions and explore issues from new perspectives.

This week’s paper presents the results of a survey containing responses from 19 software development teams about their retrospective meeting practices, and how their teams gather and use subjective and objective data in their retrospectives.

Most teams participating in the survey conduct retrospectives every two weeks, with meetings generally lasting 60 minutes. Teams vary in size (5–16 members) and usually conduct their retrospectives remotely or online (side note: Traditional in-person retrospective meetings appear to be a thing of the past, at least in this sample.). Most respondents see retrospectives as highly valuable for improving team collaboration but have varied opinions about their perceived impact on the actual work being done.

A thematic analysis of the survey results yields eight themes related to current retrospective practices:

1. The retrospective starts before the meeting itself. This includes setting up tools, gathering metrics, and allowing team members to add discussion topics in advance.

2. Custom retrospective boards and different tools are used to facilitate structured conversations and capture discussion points.

3. Some teams have introduced more casual or creative elements into their retrospectives, such as fun metaphors or non-traditional structures.

4. Retrospectives are used to support just-in-time learning, which in this context seems to be business jargon for “learning from mistakes”.

5. Several respondents believe retrospectives to be important but find their current execution to be lacking and possibly even a waste of time.

6. Teams experiment with restructuring and breaking into smaller groups for more focused meetings, occasionally resulting in actual organisational changes.

7. Some teams include a lot of operational metrics in their retrospectives, but most prioritise open discussions and personal reflections.

8. Teams focus mostly on empathy, well-being, and workplace satisfaction over purely data-driven processes.

When it comes to challenges and barriers to using objective data in retrospectives, four more themes emerge:

1. The issue is not so much that teams don’t have access to detailed metrics but that some teams simply prefer to rely on simpler tracking methods such as boards or spreadsheets.

2. Teams track a variety of metrics, but they are not integrated into regular processes like sprint planning or retros, limiting their practical use.

3. Data access and usage differ between roles. Managers and team leads are more likely to use data for decision making than the broader team.

4. Although respondents see the value in metrics, teams tend to approach them defensively, treating them as a way to justify their performance rather than as a tool for growth.