The product backlog

A product backlog (or simply “backlog”) is a prioritised list of work items that agile teams use to coordinate work to be done. For many teams it’s one of the most important project management artefacts: whoever fills and prioritises the backlog gets to determine what the team will work on.

According to the Scrum guide, a backlog is supposed to be an ordered list of everything that is known to be needed in the product. Although countless books and blogs have been written about backlogs, surprisingly little empirical research exists on what backlogs actually look like. This paper therefore tries to answer the ag(il)e-old question what product backlogs are, what their roles are, and how they emerge.

The authors of this paper conducted a longitudinal study at Pivotal, a large international software company with twenty offices around the world, over a period of more than two years. The scope of the study included eight greenfield and brownfield Extreme Programming projects from various domains.

Data were collected using a variety of methods: participant observation, interviews, questionnaires, and analysis of organisational artefacts. All of this was basically done until the researchers reached the saturation point at which no new findings emerged.

The findings provide insights into observed product backlog practices and obstacles that are regularly encountered by study participants. These are then abstracted into a more transferable theory of the backlog.

The researchers observed 13 practices related to backlog generation and refinement.

Pivotal uses two important organizational systems: “balanced teams” for people and “dual-track agile” for activities.

• Balanced teams: Teams should include members from diverse backgrounds, who combine business knowledge (product managers), design skills (product designers), and technical skills (engineers). If a role is under- or over-represented, the team becomes unbalanced and the product may suffer.

• Dual-track agile: Team activities are organised into two parallel tracks. Track 1 includes research, negotiation with stakeholders, and other activities that help the team determine what to work on. Track 2 involves building, testing, deploying, and so on.

  The whole team is involved in both tracks, but designers focus on track 1, while engineers focus on track 2. Ideally, track 1 stays ahead so engineers always know what to do next.

  A backlog helps the team to connect these two tracks.

A persona is a description of a fictitious user based on data from user research.

• Persona modelling: Designers use proto-personas (personas that are simply made up rather than based on data) to represent a group of similar users whose needs and behaviours need to be taken into account.

• Stakeholder mapping: Product designers lead the team in drawing a map of a product’s stakeholders, especially those within the client organisation (side note: Pivotal works with clients to transform their engineering cultures while building products.).

• Interviews: Product designers interview clients, current users of a system, or people in a particular demographic who might use the system. A few team members attend interviews and write key ideas on sticky notes. The purpose of the interviews is to understand the interviewee, not to elicit requirements or discover solutions – that all comes later.

  Designers typically start with about five interviews per proto-persona and continue until they don’t learn new things any more. Insights gained from the interviews help designers to refine the proto-personas into user personas. These personas can later be used to validate new products, features, or solution concepts.

• Affinity mapping: Teams write down new discoveries on sticky notes, which are then clustered (and occasionally rearranged) on a large whiteboard or wall. New design insights are recorded on a new sticky note and added to a special “design insights” list.

  Study participants see adding customer requests directly to the backlog as an anti-pattern, as it creates a Frankenstein amalgam of features.

Once designers feel they have a reasonably good understanding of user needs, they start to generate solution concepts.

• Design studio: Product designers would often be seen leading the team in a so-called “design studio”, in which the team iteratively generates and discusses design ideas.

• Sketching mockups: Team members iteratively sketch mockups of pages or screens that show how users would interact with them, until they converge on a promising solution concept.

• Usability testing/validation testing: Meanwhile, designers iterate between refining mockups and conducting further user interviews. In these interviews the designers conduct “usability testing”, which helps them validate or invalidate each feature represented by the mockups.

  As the product and the team’s understanding of users mature, designers spend less time on exploratory research and more on validation and usability research, and rely more on telemetry from working software.

• Writing user stories: Product managers write user stories, i.e. brief informal descriptions of some aspect of a software system. These stories are the smallest possible features that still provide user value and are based on mockups, discussions with product designers, and their own research. Most stories include a motivation (“As a <user> I want to <action> so that <value>”) and acceptance criteria (“Given <context>, when <condition> then <action>”).

Once designs have been made and stories have been written, it’s time to generate backlog items.

• Story showcase: Teams hold story showcase meetings, in which the product manager and designers share their understanding of the stories with engineers. Stories are revised, combined or broken up. Moreover, acceptance criteria are discussed and clarified.

• Backlog grooming is the process of reviewing and refining a backlog. Product managers ensure that stories are both clear and complete, and ordered in the right sequence.

• Accepting stories refers to the process in which product managers evaluate delivered work by verifying that implemented stories function according the acceptation criteria. Stories that do not are rejected, necessitating rework.

Teams routinely encountered obstacles due to stakeholders that misunderstand the nature of product design. Some specific examples include:

• Preconceiving problems: People often assume that problems to be solved are simply known or given, but in reality, stakeholders often disagree on the exact nature of the problem(s). It’s the designer’s job to frame the problem and generate solution concepts. Overspecifying problems for the designer impedes effective solution generation.

• Preconceiving solutions: Even worse are situations in which a client arrives with a dubious problem and an ill-considered solution concept. Good solution concepts only emerge from a complicated process that involves stakeholders, thorough research, and reflection.

• Pressure to converge: Creative design is closely related to divergent thinking, i.e. the exploration of many different ideas. If a team doesn’t thoroughly explore the design space (side note: This may take days or even weeks!), it will likely converge prematurely on a poor solution concept.

• Ambiguity: Product managers, engineers and other stakeholders often struggle with ambiguity, even though it’s a natural part of the design process.

  A common sequence that was observed by the researchers begins with a client that arrives with some idea about the “problem” and a possible solution, who expects the team to refine and elaborate them. Instead, user research undermines their preconceptions before offering new insights.

  Product designers should help stakeholders understand the process and reassure them that ambiguity is normal.

• Time pressure: Research and design take a lot of time. Product designers often feel pressure to cut corners, especially when the client doesn’t understand (the value of) the design process or believes they have already designed a product. This leads to flawed products and consequently substantial stress and rework.

• Blocking access to users: In one extreme case, the product team was not given access to prospective users. This made it impossible to develop realistic personas and validate any of the solutions that were proposed by the team.

As we’ve seen above, teams often organise their activities into two tracks, which are separated by a clear boundary: the product backlog.

The product backlog is an informal model of work to be done. The ones that were observed are partially-ordered lists of brief work item descriptions:

• Only some items include effort estimates and other metadata.

• Only immediate work items (those near the top) may be sequenced. The rest is mostly used by the product manager to externalise thoughts about features, and by the team to remember what to do next, drill down into specifics or recall details of a discussion.

The backlog facilitates knowledge sharing and coordination between the product manager, product designers and the team. Backlog grooming and story showcase meetings help build a shared understanding of the product vision and work to be done.

Finally, backlogs appear to emerge from three interconnected processes:

1. Sensemaking: Product designers deploy several practices (stakeholder mapping, interviewing, persona modelling and affinity mapping) to learn about the domain.

2. Context-solution coevolution: Product designers iteratively evolve solution concepts while they try to gain an even better understanding of the problem. This is done together with teams, who create mockups, conduct usability tests, and write user stories.

3. Boundary spanning: Product managers bridge the gap between track 1 (led by designers) and track 2 (led by engineers) using backlog grooming, story showcasing, and accepting stories.