When not to comment: Questions and tradeoffs with API documentation for C++ projects (2018)

A very simple to use interface that is accompanied by a way too elaborate manual, and a complex interface that only comes with an empty sticky note

Many developers dislike writing documentation. However, if someone else’s code isn’t sufficiently well-documented, they’re often the first to complain about it. Thanks to this study by Head, Sadowski, Murphy-Hill, and Knight, we now know how much you should invest in API documentation (and how much you can get away with).

Why it matters

Documentation can be flawed for many different reasons: it might be out-of-date, poorly written, overwhelming, contain errors, or even not exist at all. It’s therefore not completely surprising that more experienced developers often choose to read (comments in) the source codeOf course, this assumes that you can access the source code in the first place. If you can’t, then… tough luck instead.

That’s not a good thing: it implies that lot of time is wasted by developers who spend their precious time on writing comments that won’t actually be read, and developers that mentally “re-engineer” code because they purposefully choose to ignore well-written comments. Does it still make sense to carefully document everything then?

How the study was conducted

The study was conducted at Google, which has an interactive code search tool for all of its code. The tool helps their developers find answers to questions related to API usage by allowing them to search within and navigate between files.

The authors used the tool to reach two groups of developers:

Searchers who used the tool to navigate from header to implementation filesHeaders files describe the interface of a class, while the implementation files contain the “actual code”. This makes it possible to reliably infer which searchers shift their focus from API documentation to (comments within) source code were asked to complete a brief survey about why they chose to do so. Interviews were also held with searchers and maintainers to better understand their perspectives on what should and what shouldn’t be documented.

What discoveries were made

The results are presented separately for each developer type.

What searchers think

Searchers generally expect to find answers to API-related questions in header files: this is often the case with popular, high-quality APIs, which tend to be well-documented. Comments in the header files may include helpful suggestions that document best practices, explain what a method does, how it works, and how it should be called.

Unfortunately, documentation can also be inaccurate, incomplete, incorrect, or simply not exist. That’s why some searchers prefer to read the code instead, which is somewhat guaranteed to provide the correct answers to any technical questions the searcher may have. This works especially well if the code is simple: comments rarely offer any advantage over source code in such cases.

What maintainers think

Maintainers generally believe that documentation should only be added when it’s really necessary. Methods with obvious signatures should not be documented, and searchers should have sufficient background knowledge to understand non-functional aspects of API selection and usage – otherwise they are not part of the API’s target audience.

Comments also shouldn’t be written if there is no intention of maintaining the code, if the code is still actively being evolved or fixed, or when good enough documentation exists for similar external projectsExamples include reimplementations of existing libraries or new implementations of previously documented interfaces.

Finally, maintainers attempt to preserve the existing comment style when adding new comments to an existing codebase, and emphasise the importance of writing comments that aren’t likely to become outdated.

The important bits

Stick to these guidelines when documenting code:

  1. Documentation rots faster than code, so don’t write too much of it – if you do write a comment, make sure it can stand the test of time
  2. Document only the functionality of a method and the way it should be used, and ignore non-functional aspects
  3. There’s no need to document code that’s self-explanatory, but code that may be hard to understand should be commented
  4. Comments preferably also convey high-level information about things like recommended usage and context of an API (method)