On the usage of Pythonic idioms

Design patterns are conventions that make it easier to communicate and reason about commonly occurring problems and their solutions. The Python community takes the idea of using conventions one step further, and argues that code should be “Pythonic”. Alexandru et al. explore what that means in practice.

The Python community cares so much about having a single “right” way to design and implement solutions, that it has even coined a special word for it: “Pythonic”.

The term appears to be very widespread: the word “Pythonic” is mentioned in a staggering 90% of all Stack Overflow threads about Python.

Nevertheless, the concept itself has never been thoroughly researched.

The authors aim to learn how well the concept of “Pythonic” code is understood by developers and how widespread concrete Pythonic idioms are.

First, they interviewed 13 Python developers with diverse backgrounds and levels of Python experience.

Then, the authors compiled a list of idioms by manually analysing presentations, books, and online resources by renowned Python developers (side note: More info in the box below!).

Each of these idioms can be represented using an abstract syntax tree (side note: An abstract syntax tree shows the “shape” of the solution without considering things like whitespace, newlines, and variable names.). The authors therefore mined 1,000 popular Python repositories on GitHub for these abstract syntax trees to determine how prevalent these idioms are in practice.

Correct usage of Pythonic idioms is considered to very important and sets expert Python developers apart from less experienced Python developers.

All interviewed developers were familiar with the term “Pythonic” and agreed that Pythonic code is desirable.

Pythonic code is generally understood to be elegant and readable code that makes good use of features offered by the language and its standard libraries. This often results in more succinct code, improved readability, and better performance.

Developers usually learn about the best way to write Python code by reading books and code written by others, attending conferences, discussions with colleagues, and consulting Stack Overflow.

Actually applying this knowledge to new code can be hard at first though, and may require years of practice before one can readily implement them. This is probably why the “Pythonic-ness” of code can be used to estimate its developer’s level of Python expertise.

Writing pythonic code is usually mandatory in open-source projects. However, code in closed-source projects often has lower standards and is more likely to be “merely” documented (side note: It’s not clear whether this means that developers working on closed-source projects are less skilled in Python or that it’s harder to accurately estimate skill levels for developers who work on closed-source projects.).

The table below shows the idioms that were found in the 1,000 projects, together with the number of projects in which they were used and the total number of usages across those projects.

Idiom	# Projects	Total
List comprehension	866	75,466
Dict comprehension	146	796
Generator expression	709	33,038
Decorator	765	114,545
Simple magic methods	759	78,376
Intermediate magic methods	417	13,255
Advanced magic methods	190	2,613
finally	504	18,900
with	848	143,453
enumerate	681	19,453
yield	672	56,687
lambda	664	109,600
collections.defaultdict	314	2,930
collections.namedtuple	262	2,211
collections.deque	180	1,698
collections.Counter	133	1,074
@classmethod	518	22,220
@staticmethod	487	11,552
zip	554	14,929
itertools	128	839
functools.total_ordering	30	82

You can find more thorough explanations of these idioms on the paper’s companion website.