A theory of Scrum team effectiveness

Scrum is the most commonly used agile framework for software teams. Although the methodology has been studied quite often in the past few decades, much of what we know about how Scrum works in practice comes from field studies, which are not generalisable.

This week’s paper describes a large-scale cross-sectional study on Scrum teams and their internal dynamics; what are the key factors of effective Scrum teams and how do they interrelate?

The study consists of two parts. In the first part, the researchers analysed 13 case studies between 2014 and 2019 in order to develop a theory of Scrum effectiveness. This theory is then statistically assessed using a quantitative survey that was completed by about 2,900 respondents across almost 1,200 teams.

The researchers initially identified four key factors for effective Scrum teams: concern for stakeholder needs, responsiveness, continuous improvement, and team autonomy. Several new factors were added during analysis.

Scrum teams varied in the degree to which the needs of external stakeholders (customers, key users, investors) were considered in their decision making.

• Value focus: Not all Scrum teams focussed equally on the value of their work. While some product owners frequently discussed with their team how work fit within the bigger picture and how it impacts stakeholders, others simply pass requirements on without elaboration.

  Product backlog items are also described in different ways. Some capture work in pure technical terms, while others also clarify how the item connects to needs of stakeholders.

• Stakeholder collaboration: Some teams frequently invited stakeholders to sprint reviews. In many teams developers also interact directly with stakeholders, while in some cases they also trained stakeholders to use their products. In some teams only the product owner interacted with stakeholders.

• Sprint review quality: The sprint review is supposed to be an event where feedback is gathered from stakeholders and adjustments are identified. This is not always the case in practice. Some teams never invite stakeholders. The format also differs, from formal presentations to hands-on demonstrations.

• Shared goals: Shared goals encourage collaboration and clarify how tasks relate to the overall business goal. In some teams, the product owner begins sprint planning by stating a business objective that guides the ordering and selection of work, while in others teams simply select the top items from the product backlog.

The observed teams had varying degrees of freedom in how they performed their work:

• Self-management: While all teams were able to distribute work internally as they saw fit, not all teams had control over incoming work and when to perform it. Product owners do not always have autonomy to make decisions.

• Cross functionality: Some teams cover a diverse range of skills from testing, business analysis, and design to coding. However, there are also teams in which the range of skills was much narrower, often only coding.

Differences in continuous improvement manifested themselves in five areas:

• Retrospective quality: The sprint retrospective is a recurring event where teams can plan ways to increase quality and effectiveness. Most teams did this every sprint, although there were also teams that did so less frequently. Many teams used different themes. Only a small number of teams repeated the same format (“tips and tops”). A more diverse range of formats and themes tends to generate more improvements in a broader range of areas.

• Quality concern: Some teams frequently kept their definition of done up-to-date, while others never or rarely did. Moreover, not all teams explored the use of new practices and technologies to improve quality.

• Psychological safety: Teams diverged in how they addressed internal conflicts. Patterns of interrupting and talking over each other could be seen in some teams. At the same time, there were also teams that purposefully made time for members to express personal concerns and frustrations.

• Shared learning: Some technical and process improvements transcend teams. Some teams share their learning with other teams, invite management, work on shared challenges, or start communities of practice. Others generally don’t.

Scrum teams varied in their ability to respond quickly to emerging needs or issues:

• Release frequency: Scrum teams are supposed to deliver at least one increment every sprint that is either released or ready for release. Only two teams managed to do this. Most teams only released intermediate versions to staging environments or needed several sprints to produce a releasable increment. Releases therefore often happen infrequently, and only once per quarter or less.

• Refinement: Teams use refinements to break down large features into smaller chunks that can be delivered within the scope of a sprint. Most teams do this during weekly workshops, some during sprint planning. Funnily enough, teams consistently believe they should refine more often.

The role of managers varied greatly between cases. In some teams, managers had supporting roles, while in others management remained distant or retained a directive style.

Managers support teams in expanding their skills in several ways, e.g. training, personal development, adding members with missing skills, giving teams the freedom to change their composition as needed, or by managing the boundaries of Scrum teams.

According to teams themselves, two main variables characterised the effectiveness of Scrum teams:

• Stakeholder satisfaction: Effective Scrum would make it easier to satisfy stakeholders, because it allows the team to release increments sooner, include new ideas as they emerge, and collaborate more intensively with stakeholders.

• Team morale, as a result of increased collaboration, doing “work that matters”, “more personal autonomy”, and “getting things done”.

The findings from the multiple case study led to the formulation of six hypotheses, which together form a theoretical model. These hypotheses were validated using a quantitative survey.

The main findings are as follows:

• Scrum teams that can release to stakeholders more often are more likely to see higher stakeholder satisfaction and higher team morale. The ability to release often can be defined as “responsiveness”. It includes not only release frequency, but also the supporting skills that are needed to create this ability.

• However, responsiveness alone is not enough. Scrum teams are more likely to satisfy stakeholders if they have a strong concern for them. This also works in the opposite direction: teams that have a strong concern for stakeholders are more inclined to increase their responsiveness.

• Scrum teams that engage in continuous improvement are more likely to show greater concern for the needs of stakeholders, but curiously there is no positive effect of continuous improvement on responsiveness. This may suggest that responsiveness has more to do with technical and skill-based challenges than stakeholder concern.

• Team autonomy significantly contributes to the degree of continuous improvement and concern for stakeholders. The larger the mandate of a team, the more a product owner can capitalise on stakeholder needs that are discovered as work progresses. Moreover, autonomous teams are more inclined to take ownership of their work and the way that work is done.

• Finally, management support positively influences team autonomy, continuous improvement, and stakeholder concern: managers can help Scrum teams by removing constraints in their environment, i.e. the boundaries.

Characteristics of effective Scrum teams include: