Does goal-oriented requirements engineering achieve its goal?

Building things right is important, but so is building the right thing. There’s plenty of research that suggests that goal-based thinking helps create better requirements, but practitioners don’t appear to be entirely convinced yet. Mavin et al. tried to better understand this gap between academia and industrial practice.

Goals are aspirational, and capture what stakeholders ideally want a system to do. Requirements on the other hand define what the system should actually do. Without this distinction it’s easy to write requirements that are impossible to meet or to lose sight of what stakeholders really wanted, which leads to costly rework.

The authors therefore want to know:

• whether research on goal-oriented requirements engineering (RE) actually considers applications within industry or is more of the “ivory tower” type, and

• how requirements engineers use goals in the RE process.

The study consisted of two parts.

First, a literature study was conducted on a set of 246 highly-cited goal-oriented RE research papers that was previously compiled by Horkoff et al (2016). Each of those papers was classified according to five dimensions:

• Condition: was the study conducted within a real project or in an artificial environment?

• Purpose: did the study evaluate the feasibility or utility (side note: That is, was it a study that tries to determine whether some new approach is useful at all, or did it look at how well a previously introduced approach performs in practice?) of the goal-oriented approach?

• Involvement of practitioners: is it likely that practitioners were involved in the creation of the goal models?

• Feedback: did the authors ask for feedback from practitioners in some way or did they not ask for any feedback at all?

• Source: did the case or requirements originate from practice or from academia?

The second part of the study consisted of a questionnaire about respondents’ understanding of goals and their use of goals in their daily work. This questionnaire was sent out to 46 RE practitioners from the authors’ personal networks.

The results suggest that there’s indeed a gap between industry and academia.

The literature study shows that the majority of papers are research-oriented: 92% of authors have an academic affiliation, while 86% of papers did not involve any (co-)author from industry or an applied research institute.

There were 120 papers that described case studies, of which 76.7% were intended to determine the feasibility of some new goal-oriented approach, while only 23.3% looked at utility of existing approaches.

The origin of case studies is evenly divided, but a very large majority (83.3%) was conducted in artificial conditions rather than real-world settings.

Most studies did not elicit any feedback from RE practitioners at all.

This all suggests that RE research is mostly “by academia, for academia”.

I’ll only mention observations that I found interesting here; you can find a complete discussion of each question in the original paper.

The results show that even though many respondents do think in terms of goals and requirements, there’s little consensus about what a goal exactly is and how goals should be used.

What’s also surprising is that about two-thirds of respondents are aware of formal goal modelling approaches (side note: Examples include i* and KAOS), but few actually use them.

Respondents that use or have used formal approaches, appreciate the benefits that they can provide. However, they’re also concerned about the scalability of the approaches and the additional overhead that’s needed to implement them effectively.

Other respondents often cite “adoption barrier” and “no need” as reasons to not use any formal approach at all. New RE research should therefore not only look at feasibility of new approaches, but also address the concerns that are voiced by practitioners.

1. Horkoff, J., Aydemir, F. B., Cardoso, E., Li, T., Maté, A., Paja, E., … & Giorgini, P. (2016). Goal-oriented requirements engineering: a systematic literature map. In 2016 IEEE 24th International Requirements Engineering Conference (RE) (pp. 106–115). IEEE.