Intrusion of software robots into journalism

Good journalism is exciting, inspiring, and informative. But there’s also a lot of mundane reporting that simply isn’t very exciting to produce. One might be tempted to automate these tasks. Are articles written by robots robust enough to withstand the watchful eyes of journalists and the general public? Jung et al. think that may be the case!

Let’s start with a simple question. Are algorithms able to generate trivial articles (side note: i.e. articles that can be generated automatically from structured data using pre-determined sentences) that are good enough?

I’ll just spoil this one right away, because it’s not part of the main study: the quality of articles written by algorithms can be indistinguishable from that of articles that were written by journalists, if one leaves out the byline (side note: i.e. the part that shows the author’s name).

But what happens when we do add that byline?

Trust in mainstream media – and consequentially journalists who work for those media – has slowly decreased among the general public in the recent years. It’s therefore probable that the general public will be more favourable towards articles that are written by algorithms.

Journalists on the other hand, might be worried about the possibility that robots will take over their jobs and therefore prefer articles that have been written by a human peer.

These are assumptions that can be tested.

The authors performed two studies, one for each target group:

• 400 respondents from the general public that were recruited from a Korean online panel;

• 164 professional journalists that were recruited via snowball sampling (side note: This is a sampling technique where each participant is asked to recruit more participants. Not to be confused with “snowballing” (don’t google that at work).).

In each study participants were asked to read an article that was written by either a robot or journalist, and had a byline of either a robot or journalist, regardless of who wrote the article. Afterwards, participants were asked to answer a series of questions about the article on a 5-point Likert scale; was it well-written, clear, professional, and credible?

The table below shows the average ratings given by respondents from the general public (higher is better):

	Written by robot	Written by human	Avg.
Robot in byline	3.39	3.41	3.40
Human in byline	3.21	3.31	3.26
Average	3.30	3.36	3.33

Respondents think that the quality of the hand-written article is slightly higher than that of the algorithmically generated article. However, we can also see that articles that have a byline that claims that it’s written by a robot are perceived to be of higher quality – regardless of who actually wrote it!

How did the journalists rate the articles then?

	Written by robot	Written by human	Avg.
Robot in byline	3.23	3.33	3.28
Human in byline	2.85	3.13	2.99
Average	3.04	3.23	3.14

The same pattern is visible here as well: putting a robot in the byline appears to have a positive effect on the perception of the quality of an article (side note: In both cases the best-rated article is hand-written, but attributed to a robot. One might be tempted to exploit this – don’t.).

It’s not clear why this happens. Maybe journalists don’t feel that much (or any) hostility towards algorithms; maybe Koreans are more receptive of technological innovations; or maybe people simply have lower expectations of algorithmically generated articles.

Well, that’s something for another study!