When not to comment: Questions and tradeoffs with API documentation for C++ projects

Many developers dislike writing documentation. However, if someone else’s code isn’t sufficiently well-documented, they’re often the first to complain about it. Thanks to this study by Head, Sadowski, Murphy-Hill, and Knight, we now know how much you should invest in API documentation (and how much you can get away with).

Documentation can be flawed for many different reasons: it might be out-of-date, poorly written, overwhelming, contain errors, or even not exist at all. It’s therefore not completely surprising that more experienced developers often choose to read (comments in) the source code (side note: Of course, this assumes that you can access the source code in the first place. If you can’t, then… tough luck) instead.

That’s not a good thing: it implies that lot of time is wasted by developers who spend their precious time on writing comments that won’t actually be read, and developers that mentally “re-engineer” code because they purposefully choose to ignore well-written comments. Does it still make sense to carefully document everything then?

The study was conducted at Google, which has an interactive code search tool for all of its code. The tool helps their developers find answers to questions related to API usage by allowing them to search within and navigate between files.

The authors used the tool to reach two groups of developers:

• Searchers: C++ developers who have API-related questions and hope that the tool can help them answer those questions;

• Maintainers: C++ developers who have recently worked on methods that are “popular” amongst searchers/

Searchers who used the tool to navigate from header to implementation files (side note: Headers files describe the interface of a class, while the implementation files contain the “actual code”. This makes it possible to reliably infer which searchers shift their focus from API documentation to (comments within) source code) were asked to complete a brief survey about why they chose to do so. Interviews were also held with searchers and maintainers to better understand their perspectives on what should and what shouldn’t be documented.

The results are presented separately for each developer type.

Searchers generally expect to find answers to API-related questions in header files: this is often the case with popular, high-quality APIs, which tend to be well-documented. Comments in the header files may include helpful suggestions that document best practices, explain what a method does, how it works, and how it should be called.

Unfortunately, documentation can also be inaccurate, incomplete, incorrect, or simply not exist. That’s why some searchers prefer to read the code instead, which is somewhat guaranteed to provide the correct answers to any technical questions the searcher may have. This works especially well if the code is simple: comments rarely offer any advantage over source code in such cases.

Maintainers generally believe that documentation should only be added when it’s really necessary. Methods with obvious signatures should not be documented, and searchers should have sufficient background knowledge to understand non-functional aspects of API selection and usage – otherwise they are not part of the API’s target audience.

Comments also shouldn’t be written if there is no intention of maintaining the code, if the code is still actively being evolved or fixed, or when good enough documentation exists for similar external projects (side note: Examples include reimplementations of existing libraries or new implementations of previously documented interfaces).

Finally, maintainers attempt to preserve the existing comment style when adding new comments to an existing codebase, and emphasise the importance of writing comments that aren’t likely to become outdated.