Cargo cults in software engineering

Lightweight agile software development methodologies, like Scrum and Extreme Programming, are easy to follow, flexible, and easier to adopt than traditional methodologies, like RUP and DSDM.

Agile methodologies often lack clear guidelines and direct instructions on how to develop software. Consequently, different teams often have their own distinct interpretations and implementations of these methodologies.

Some of these agile teams are very successful, while others appear to struggle despite having implemented agile in some way. Why does this happen? A possible explanation is that some teams exhibit “cargo cult” behaviour, i.e. they blindly perform activities without following the underlying values and principles of the agile manifesto. This is often referred to as “doing agile, instead of being agile”.

Cargo cult behaviour was first identified among natives in Melanesia and is often used to describe situations in which a technologically less advanced culture makes futile attempts to imitate western visitors in the hope that “gods” will reward them with gifts.

Both during and after World War II the Melanesian natives saw many aeroplanes that airdropped cargo for Japanese and (later) American soldiers, who then shared some of those goods with the local population.

The natives believed that the goods were sent to the soldiers as a reward for having constructed airfields. In an attempt to please the gods and lure more of them to the island, the natives constructed their own airfields in the dirt, complete with aeroplanes made out of palm leaves. But no planes would ever touch ground on the natives’ airfields: the natives had simply replicated what they observed without having understood the underlying reasons and requirements for an airfield.

Cargo cult behaviour can also be found in software development, for instance when developers follow agile rituals without understanding the rationale for those rituals:

• The daily scrum is a time-boxed daily meeting that’s held to synchronise the team’s efforts and plan the work for the next 24 hours. If the meeting is only used to provide status updates and no planning is ever done, it’s arguably just a glorified coffee break.

• The product backlog is a list of work items that have been estimated and prioritised, so that the team knows what it will work on in the next iteration. If work items are not described, prioritised or estimated, the team could be focussing on the wrong work items that provide limited or no value to the customer.

So far, research has mainly focussed on the identification of Scrum/agile anti-patterns and possible remedies (side note: I wrote about a paper that discusses Scrum anti-patterns a few years ago.), but not on what causes these anti-patterns to appear in the first place.

Cargo cult behaviour is typically carried out within a team and can thus be understood as a (flawed) social action, which has previously been described in literature as follows:

• A traditional action is behaviour that’s governed by habits and no longer takes rational reasons into consideration. A developer does things the way “it’s always been done” and doesn’t think about whether it can be done differently.

• An affectual action is based on a person’s current emotions; they might become inspired and emotionally embrace ideals without having really understood them.

• Value-based actions are based on an irrational ambition to strive towards and fulfil certain intrinsic motives. As a result, someone who carries out a value-based action does not think about the consequences of that action.

• Means-end actions are carried out to achieve specific extrinsic goals. After rational consideration of different alternative means and ends, a person makes a deliberate choice for a specific activity to reach a calculated end.

Based on the above typology of social action, an analytical framework was developed that helps us explore the phenomenon of cargo culting:

	Intrinsic	Extrinsic
Non-rational	Affectual Hype Bandwagon effect	Traditional Keeping bad habit Old wine in a new bottle
Irrational	Value Ignorant of practices	Means-end Malpractice Replication of practices

In this framework there are two dimensions that further clarify the nature of cargo cult behaviour:

• The possible existence of rational thought in relation to the rationale for a part of a software development methodology. Cargo cult behaviour could arise from a misunderstanding of the reasons behind the methodology or involve no reasoning at all.

• The motivation behind why something is done, which can be either intrinsic (for its own sake) or extrinsic (to achieve a goal).

When combined, these two dimensions result in four types of cargo cult situations; one for each type of flawed social action:

• Affectual: A team “jumps on the bandwagon” or follows the “hype”, without considering why something is done or with a misconception of what really needs to be done and how much effort it would take.

• Value: In this case, a team embraces the true meaning behind the agile values, but lacks the necessary know-how to implement agile in a proper way and consequently does not look at the intended purpose of agile activities.

• Means-end: The team seeks to achieve certain outcomes (e.g. faster deliveries, better communication with customers) and tries to perform actions that should help them achieve their goals. However, they do this based on wrong interpretations of how something is supposed to be done, or simply imitate the behaviours from another successful organisations without taking context into account (side note: bUt GoOgLe DoEs It ToO!1!1).

• Traditional: The team has adopted some, but not all agile practices. Some old habits have been unintentionally been kept from an older methodology. No one questions these old habits, as they were useful in the past.