Chuniversiteit logomarkChuniversiteit.nl
The Toilet Paper

Intrusion of software robots into journalism

Robots might take over jobs in journalism. What else is new(s)?

Clippy attends a press conference
All your pays are belong to us

Good journalism is exciting, inspiring, and informative. But there’s also a lot of mundane reporting that simply isn’t very exciting to produce. One might be tempted to automate these tasks. Are articles written by robots robust enough to withstand the watchful eyes of journalists and the general public? Jung et al. think that may be the case!

Why it matters

Link

Let’s start with a simple question. Are algorithms able to generate that are good enough?

I’ll just spoil this one right away, because it’s not part of the main study: the quality of articles written by algorithms can be indistinguishable from that of articles that were written by journalists, if one leaves out .

But what happens when we do add that byline?

Trust in mainstream media – and consequentially journalists who work for those media – has slowly decreased among the general public in the recent years. It’s therefore probable that the general public will be more favourable towards articles that are written by algorithms.

Journalists on the other hand, might be worried about the possibility that robots will take over their jobs and therefore prefer articles that have been written by a human peer.

These are assumptions that can be tested.

How the study was conducted

Link

The authors performed two studies, one for each target group:

  • 400 respondents from the general public that were recruited from a Korean online panel;

  • 164 professional journalists that were recruited via .

In each study participants were asked to read an article that was written by either a robot or journalist, and had a byline of either a robot or journalist, regardless of who wrote the article. Afterwards, participants were asked to answer a series of questions about the article on a 5-point Likert scale; was it well-written, clear, professional, and credible?

What discoveries were made

Link

The table below shows the average ratings given by respondents from the general public (higher is better):

Written by robot Written by human Avg.
Robot in byline 3.39 3.41 3.40
Human in byline 3.21 3.31 3.26
Average 3.30 3.36 3.33

Respondents think that the quality of the hand-written article is slightly higher than that of the algorithmically generated article. However, we can also see that articles that have a byline that claims that it’s written by a robot are perceived to be of higher quality – regardless of who actually wrote it!

How did the journalists rate the articles then?

Written by robot Written by human Avg.
Robot in byline 3.23 3.33 3.28
Human in byline 2.85 3.13 2.99
Average 3.04 3.23 3.14

The same pattern is visible here as well: .

It’s not clear why this happens. Maybe journalists don’t feel that much (or any) hostility towards algorithms; maybe Koreans are more receptive of technological innovations; or maybe people simply have lower expectations of algorithmically generated articles.

Well, that’s something for another study!

Summary

Link
  1. Articles that are claimed to have been written by an algorithm are perceived to be of higher quality, regardless of the actual author