Does goal-oriented requirements engineering achieve its goal? (2017)

A football player that doesn’t shoot balls into the goal

Building things right is important, but so is building the right thing. There’s plenty of research that suggests that goal-based thinking helps create better requirements, but practitioners don’t appear to be entirely convinced yet. Mavin et al. tried to better understand this gap between academia and industrial practice.

Why it matters

Goals are aspirational, and capture what stakeholders ideally want a system to do. Requirements on the other hand define what the system should actually do. Without this distinction it’s easy to write requirements that are impossible to meet or to lose sight of what stakeholders really wanted, which leads to costly rework.

The authors therefore want to know:

How the study was conducted

The study consisted of two parts.

First, a literature study was conducted on a set of 246 highly-cited goal-oriented RE research papers that was previously compiled by Horkoff et al (2016). Each of those papers was classified according to five dimensions:

The second part of the study consisted of a questionnaire about respondents’ understanding of goals and their use of goals in their daily work. This questionnaire was sent out to 46 RE practitioners from the authors’ personal networks.

What discoveries were made

The results suggest that there’s indeed a gap between industry and academia.

Literature study

The literature study shows that the majority of papers are research-oriented: 92% of authors have an academic affiliation, while 86% of papers did not involve any (co-)author from industry or an applied research institute.

There were 120 papers that described case studies, of which 76.7% were intended to determine the feasibility of some new goal-oriented approach, while only 23.3% looked at utility of existing approaches.

The origin of case studies is evenly divided, but a very large majority (83.3%) was conducted in artificial conditions rather than real-world settings.

Most studies did not elicit any feedback from RE practitioners at all.

This all suggests that RE research is mostly “by academia, for academia”.

Questionnaire

I’ll only mention observations that I found interesting here; you can find a complete discussion of each question in the original paper.

The results show that even though many respondents do think in terms of goals and requirements, there’s little consensus about what a goal exactly is and how goals should be used.

What’s also surprising is that about two-thirds of respondents are aware of formal goal modelling approachesExamples include i* and KAOS, but few actually use them.

Respondents that use or have used formal approaches, appreciate the benefits that they can provide. However, they’re also concerned about the scalability of the approaches and the additional overhead that’s needed to implement them effectively.

Other respondents often cite “adoption barrier” and “no need” as reasons to not use any formal approach at all. New RE research should therefore not only look at feasibility of new approaches, but also address the concerns that are voiced by practitioners.

The important bits

  1. Requirements engineering practitioners agree that goal-oriented approaches provide value
  2. There’s little consensus about what a “goal” exactly is and how it’s supposed to be used in requirements engineering
  3. Most requirements engineering research appears to be performed without any real-world grounding
  4. Solutions proposed by academia are rarely adopted in industry due to lack of confidence, concerns about overhead, and other barriers

References

  1. Horkoff, J., Aydemir, F. B., Cardoso, E., Li, T., Maté, A., Paja, E., … & Giorgini, P. (2016). Goal-oriented requirements engineering: a systematic literature map. In 2016 IEEE 24th International Requirements Engineering Conference (RE) (pp. 106–115). IEEE.