Software development waste

It’s not always easy to eliminate waste in software development, not least because it can be hard to identify. Sedano, Ralph, and Péraire collected well over two years’ worth of data from interviews and observations at Pivotal, and discovered nine types of waste in software development.

In the second half of the 1940s, the Toyota Production System gave rise to the lean manufacturing method, which heavily emphasises waste elimination.

Lean has since been “ported” to software development as Lean Software Development. However, it’s not clear how well the waste categories in lean manufacturing translate to software development, which has very different characteristics.

It therefore makes sense to empirically validate the Lean Software Development framework by independently identifying the different types of waste in software development using an in-depth study.

The first author collected data from 8 of Pivotal Labs’ software development project teams over a 29-month period using various methods:

• Participant observation, while working as an engineer at each of the 8 project teams;

• Interviews with 33 interaction designers, product managers, and software engineers from five different Pivotal offices;

• Topics mentioned during 91 retrospection meetings.

The collected data were analysed throughout the study in order to generate and refine waste categories until no further waste-related categories could be (easily) found.

A taxonomy of waste types was constructed and compared with the categories in lean software development.

Nine types of waste were identified.

1. Building the wrong feature or product

If wrong assumptions are made during design or by the business, things might be built that no user actually needs, wants, or uses. It can be hard for practitioners to balance the needs of users with those of the business.

2. Mismanaging the backlog

If engineers working through the backlog get ahead of the product manager who’s prioritising the backlog, then “backlog inversion“ may occur, i.e. engineers work on potentially outdated, low-value or even counterproductive stories first.

For the product manager this means that they will need to find the right balance between writing enough stories to keep engineers busy and writing many stories that will never be implemented.

Waste can also occur if two engineers inadvertently work on the same story due to miscommunication, if engineers are asked to work on too many features at the same time, or if the focus is constantly shifted by the product manager.

3. Rework

Rework is work that could have been done right the first time, but wasn’t. Causes for rework include: technical debt, poor testing strategies, rejected stories (side note: When the product owner rejects an implementation because it doesn’t satisfy all acceptance criteria), unclear stories, and unclear mock-ups.

4. Unnecessarily complex solutions

A feature that’s unnecessarily complex waste users’ and engineers’ time as they struggle to understand and work with it.

Lack of reuse (both on a design and on an implementation level) can also result in additional complexity.

In practice, there’s often a tension between doing big design up-front and incremental design: the former tends to result in more elegant designs, but can also produce incorrect or outdated assumptions, while the latter is more likely to lead to feature complexity and duplication.

5. Extraneous cognitive load

Overcomplicated stories, ineffective tooling, technical debt, and multitasking all can lead to extraneous cognitive load: cognitive load that’s not caused by the complexity of the actual task itself, but by the environment or the way the task is presented. Cognitive overload inhibits learning and problem solving.

6. Psychological distress

There’s good stress (eustress) and there’s bad stress (distress). Distress can lead to things like absenteeism, burnout, lower productivity, and health problems.

It can occur if a team is pressured to complete work by a given date.

7. Waiting/multitasking

An engineer might wait for a product manager to clarify a story (side note: Or try to make some assumptions, which possibly leads to rework), which is a waste of time.

Waiting is a waste that’s not always easy to spot, as people generally don’t actually wait (by doing nothing), but multitask e.g. an engineer might fill their time with a lower-priority story instead, and a product manager might work on new stories if the acceptance environment temporarily doesn’t work.

These “solutions” aren’t just bad because they cause delays: they also create extraneous cognitive load – for short waits, taking a break might actually be less wasteful than doing “useful” tasks.

8. Knowledge loss

Knowledge silos can lead to knowledge loss when a team member leaves the team or the company: remaining team members could possibly waste months trying to understand the system (side note: There’s actually a word for this: software archæology) before they can work on new features again.

9. Ineffective communication

Incomplete, incorrect, misleading, inefficient communication – or even the complete absence of communication – reduces team productivity.

The table below shows the mappings between the waste categories identified in this study and those described in Lean Software Development. Several categories appear in both taxonomies, some were not present in Lean, and one was not observed in this study (in a form that would make it a separate category).

Wastes found in this study	Lean Software Development wastes
Building the wrong feature or product	Extra features
Mismanaging the backlog	Partially done work
Rework	Defects
Unnecessarily complex solutions	-
Extraneous cognitive load	-
Psychological distress	-
Waiting/multitasking	Delays Task switching
Knowledge loss	Relearning
Ineffective communication	-
-	Handoffs