Exploring ScrumBut – An empirical study of Scrum anti-patterns

Every time I interview with a company and ask how they develop software, they’ll say something along the lines of “We use Scrum”, followed by a long list of deviations that decidedly make it not-Scrum. Eloranta, Koskimies, and Mikkonen describe 14 of these commonly used deviations and explain what makes them harmful.

Scrum is a simple, iterative framework for project management (side note: Although at times it feels awfully lot like a religion…). It helps teams deliver better software in less time. This has quickly helped it gain widespread adoption in the software industry.

Or at least, that’s what everyone wants you to believe – in practice, many companies cherry-pick the parts of Scrum that they like or that are easy to implement. The result is a process that’s no longer a comprehensive, well-thought-out framework, but a loose collection of practices that’s commonly referred to as ScrumBut.

These ScrumBut processes often have severe flaws. The authors aim to identify and describe these flaws.

The authors held semi-structured interviews with representatives from 18 teams in 11 IT companies of various sizes in Finland’s Tampere region.

Representatives included project, development, and quality managers, as well as developers. All representatives had completed a certified Scrum master course.

Smaller companies seem to follow Scrum more closely than large companies, possibly because larger companies have legacy practices that make it harder to fully embrace Scrum.

Another interesting observation is that companies that have more Scrum experience also tend to use more anti-patterns. Since process improvement is a core tenet of Scrum, it’s only natural that teams slowly deviate from “real” Scrum over time. However, one would not expect teams to adopt changes that are harmful.

The authors identified 14 harmful deviations that were observed in at least three teams and have negative consequences according to interviewees or Scrum literature.

Keep in mind that while the authors refer to these deviations as anti-patterns, the study does not take the actual impact on the development process into account.

Big requirements documentation

Scrum mandates the use of a product backlog: a prioritised wish list that contains work items in the form of small, easily understood user stories or use cases that provide value to the customer in their own right.

However, some teams still follow legacy software development guidelines, which mandate big requirements documents prior to development sprints.

Requirements in large documents are harder to write, maintain and understand for the team, and rarely prioritised (side note: This is acceptable for safety-critical systems that have strict or formal requirements.).

Customer product owner

The product owner ensures that the product produces value for the customer.

In some teams a customer representative acts as the “product owner”, because no product owner is available or the customer doesn’t trust the team. Such product owners are often too busy to prepare the backlog, are hard to reach, and primarily protect the customer’s interests rather than that of the team’s.

To solve this problem, one might create a product owner team, of which the customer representative is made a member.

Product owner without authority

The product owner is responsible for managing the product backlog and deciding which items maximise the value of the product.

If a company doesn’t understand or believe in Scrum, a team may end up with a product owner who isn’t allowed to make decisions or isn’t in contact the customer and therefore doesn’t really understand what the customer needs.

This is a problem that needs to be solved on a higher level: managers need to understand the value that a proper, empowered product owner can provide.

Long or non-existent feedback loops

Sprint reviews should be organised at the end of every sprint, so that the product owner and key stakeholders can review the work and provide feedback.

If the customer does not actively participate in the project (“I don’t have enough time”) or communicate with the product owner, it’s likely that a lot of rework will need to be done at a late stage of development.

The authors suggest that Scrum and agile development should be gently introduced to such customers; after a while they’ll usually start to like incremental development with short feedback loops.

Unordered product backlog

The product backlog must be prioritised by a product owner so that team members know which features provide most value to the customer.

If this does not happen for whatever reason, it’s likely that the team first picks up features that are low-value, fun, or easy rather than the features that the customer actually wants.

Bi-weekly product backlog grooming meetings should be organised in which the top of the backlog is prioritised by stakeholders.

Work estimates given to teams

Teams should always produce their own estimates, otherwise teams may lack commitment to achieve (possibly unrealistic) goals.

Companies in which the product owner is responsible for making estimates should try out planning poker.

Hours in progress monitoring

It’s up to the team to decide how many work items are selected from the product backlog, typically based on the number story points.

Some organisations and projects use hours for estimating or pricing features. These estimates are hard to make – especially if the user story involves knowledge gathering – and tend to be wildly inaccurate to the point where they’re essentially useless.

Semi-functional teams

Teams should have all skills that are necessary to fully realise a shippable product.

Otherwise one often ends up with staged development, where work is handed off between different teams. This results in less team commitment, lower-quality communication about important issues (e.g. user interface designs), and longer feedback loops.

Customer caused disruption

Teams should be protected against outside disruptions like phone calls, bug reports, and work for unrelated projects.

Allowing customers to contact team members at all times – or even worse: letting them negotiate about new features with the team – interrupts the team’s work flow, and might result in low-priority work being done before high-priority work.

This does not mean that communication between customers and team members is always bad of course, but it should be limited to time-boxed sessions (“question hours”).

Business as usual (no sprint retrospective)

Scrum is all about continuous improvement: after each sprint review (side note: The authors suggest that it’s okay to hold them after every other sprint if they’re only two weeks long. I disagree.) teams should hold retrospectives in which they reflect on the past sprint and create plans for improvements in the next.

Insufficient understanding of Scrum or agile may cause teams to think that they can skip doing retrospectives altogether. This makes it less likely that teams systematically learn from their mistakes and may eventually lead to abandonment of Scrum.

Invisible progress

Progress within a sprint should always be visible, preferably with burndown charts.

If the product owner only creates a burndown chart for themselves, or no burndown chart is created at all, stakeholders have no way to tell how well the team is doing. Important problems might be overlooked!

Varying sprint length

Sprints should have fixed lengths.

Without clear deadlines teams never really commit to finishing anything on time and progress becomes much harder to track; customers have no way of knowing when the new features they were promised will actually ship, while other features might not be released for a very long time, even though they’ve already been finished.

Too long sprint

Sprints should be between 2 and 4 weeks long (side note: Sprints that are less than 2 weeks in length have so much overhead that you won’t be able to get much done.).

Many teams experiment with sprint lengths, especially when there are unfinished tasks at the end of every sprint or when the customer refuses to work with short intervals. Increasing the sprint length might seem like an obvious solution then.

That’s not a good idea. Primarily because there still won’t be enough time to complete all tasks (side note: Remember Parkinson’s law? “Work expands so as to fill the time available for its completion.”), but also because increasing the sprint length reduces development predictability and flexibility.

Testing in next sprint

Each sprint should produce a new shippable product increment: anything that’s developed in a sprint also must have been tested within that same sprint.

However, some teams postpone testing until the next sprint. This can have multiple causes, e.g. waterfall-ish thinking, separate testing team, or lack of test automation.

Since new code is likely to be written on top of untested code in the next sprint, this will make it harder to fix bugs and lead to wasted effort.

Finally, the authors looked at the relationship between the 14 identified anti-patterns and the core Scrum concepts. These relationships are visualised in the graph below.

The relationship graph shows that the development team and product owner both can be affected by four different anti-patterns, which suggests that these concepts are particularly sensitive to anti-patterns.

Other concepts, like the retrospective, appear to be less sensitive to anti-patterns (side note: This does not mean that retrospectives are less risky, as the impact of even a single anti-pattern might still be very large.). The daily scrum in particular has not caused issues for any of the interviewed teams.