The complicated relationship between developers and security experts in Scrum

When it comes to cyber threats, the past month has been an interesting one. We’ve had not one but two local root escalation exploits making the rounds over the internet. For many people who work or study in higher education, ShinyHunters’ breach of Instructure, the company behind the popular Canvas learning management system, has had a massive impact on courses and exams.

Although these examples highlight that security is a topic that software developers must take seriously, the reality is that security isn’t considered a high priority by many. It also doesn’t help that security expertise is scarce.

In the following study, researchers interviewed developers and security experts to understand how they communicate and collaborate within the Scrum framework.

Most communication happens in the form of company and project guidelines, the creation of which is typically led by security experts, sometimes in collaboration with architects, CTOs, and CISOs. Although developers can often influence these guidelines, the security team will generally have the last word on them. The fact that guidelines exist doesn’t mean that they’re always followed, however, for instance due to disagreement with guidelines, time pressure, or simply a failure to prioritise security.

Alerts triggered by automated systems can often be a starting point of communication between developers and security experts. These processes are becoming increasingly automated such that tickets are automatically created for developers, removing the need for direct involvement from security experts.

Communication via a third party is also quite common, either via management, a product owner, team lead, or other technical experts who may not be part of the team, such as architects.

Direct communication between developers and security experts is rare, and often only happens when a problem cannot be solved without the expertise of the other party. When it does, it’s generally through direct messaging, ad hoc meetings, or regular meetings outside formal Scrum events such as sprint reviews and retrospectives.

Although collaboration is one of Scrum’s core principles, interviewed developers and security experts all faced the same five challenges.

The first is related to tensions and misunderstandings between the two groups. Security experts find it hard to reach out to developers, who may resist sharing information and overlook the value of the security role. Developers, on the other hand, perceive security experts as distant, unapproachable, and as authorities unwilling to compromise. Other factors include cultural differences due to different technical backgrounds and terminology, frustration among developers over time-consuming security processes, a general resistance to change, and gender challenges (side note: Men who don’t like being told what to do by a female security expert).

The second is related to management’s tendency to prioritise functionality over security. Developers do not have time to learn more about security or even to address security concerns properly. A possible problem in practice is that security experts sometimes identify issues that they believe are straightforward to solve but in reality can take weeks to implement, time which developers do not have due to planning and budget constraints. This problem is made worse by the fact that security experts are scarce and can be prohibitively expensive to employ full-time on smaller projects.

The perception of security is also seen as a major challenge. Only a few teams include security in their definition of ready and definition of done. This is not only due to the aforementioned prioritisation of functionality, but also because developers see security as an impediment that slows down progress and actively dislike working on security-related tasks.

Resource constraints are the fourth challenge. In Scrum, all skills needed to deliver a product increment should be present within the team. This is not usually the case when it comes to security. Security experts find themselves outnumbered by development teams, resulting in long waiting times for developers and high workloads (and sometimes even burnout) for security experts.

Finally, a lack of security awareness and knowledge is holding back developers from advocating for the necessary resources and understanding the impact of security measures.

The interviewees were asked about what they believed are the best ways to improve the relationship and collaboration between developers and security experts. When combined with what we already know from existing work on the subject, this yields four practical recommendations:

1. Scrum masters can play a crucial role in facilitating communication between developers and security experts. They should foster a supportive environment that enables open discussions of security issues, involve security experts in sprint planning from the start, and include them in retrospectives when security issues arise.

2. Previous work has shown that product owners often lack awareness and knowledge of security importance, and often have to prioritise functionality over security. Instead, security should be integrated into the product backlog and security experts should be invited to sprint reviews to raise stakeholder awareness and improve product security.

3. Organisations can help strengthen the relationship between developers and security experts, for example by fostering a security champions programme to improve communication about security, adopting DevSecOps to promote collaboration among development, operations, and security teams, and adopting co-creation so that developers and security experts can work side by side.

4. Organisations should develop personal relationships between developers and security experts through secure development practices (side note: The paper lists things like NIST and OWASP, but I have no idea what these have to do with personal relationships.).