Should you upgrade official Docker Hub images in production environments?

You’ve seen the title, you’ve read the caption of this article’s header image. It seems only logical that the answer is going to be “No”. And it is! Kind of. Because while this week’s paper gives us an answer, it is not the answer to the question that was asked in the title.

Docker containers consist of software code that is added to a Docker image, which provides package and library dependencies. This containerisation makes it easy to ship code to production in a repeatable and reproducible manner, by simply replacing existing, running containers with newer ones rather than deal with annoying things, like state.

Containers are not a silver bullet though: container images could have security vulnerabilities and newer versions of an image may include package changes that break existing functionality or suddenly lead to poor performance.

The authors of this paper try to learn more about package changes in Docker images, specifically the official Docker images that are published in the Docker Hub registry.

This is done using a four-step process:

1. Selecting all 158 official repositories on Docker Hub that contain downloadable images;

2. Retrieving the list of (about 37k) available tags from all repositories;

3. Downloading every image, analysing its native (operating system), Node, and Python packages, and recording when each image was last updated;

4. Identifying package changes (i.e. major, minor, and patch upgrades and downgrades) within “branches” (side note: I put this between quotes, because the concept of branches doesn’t really exist in Docker Hub, and is largely a convention that everyone just “gets”. There’s a more detailed description of the branch extraction process in the paper, if you’re interested.), for all 14 categories of official repositories on Docker Hub.

The results are not very shocking, nor is the conclusion.

Official Docker Hub images come with a median of 8.6 package upgrades. Images from repositories in the Application services category tend to have the most upgrades, followed by those from the Programming languages, Application infrastructure, and Databases categories.

Surprisingly, there is also a median of 2.1 downgrades per image, mostly in images from the Analytics category. Coincidentally, Analytics images tend to have the largest number of packages and package changes. This suggests that Analytics images are less stable than images from other categories.

Most of the changed packages are utility packages, like tzdata, base-files, libsystemd0, libudev1, and openssl. The authors argue that you do not want to upgrade utility packages unless absolutely necessary, as such upgrades might cause incompatibilities, and newer versions of these utility packages may contain bugs.

So, back to the paper’s titular question: should you upgrade official Docker Hub images in production environments? No, not without thoroughly testing them first (side note: But you were already doing this… right?).