Many developers dislike writing documentation. However, if someone else’s code isn’t sufficiently well-documented, they’re often the first to complain about it. Thanks to this study by Head, Sadowski, Murphy-Hill, and Knight, we now know how much you should invest in API documentation (and how much you can get away with).
Why it matters
Documentation can be flawed for many different reasons: it might be out-of-date, poorly written, overwhelming, contain errors, or even not exist at all. It’s therefore not completely surprising that more experienced developers often choose to read instead.
That’s not a good thing: it implies that lot of time is wasted by developers who spend their precious time on writing comments that won’t actually be read, and developers that mentally “re-engineer” code because they purposefully choose to ignore well-written comments. Does it still make sense to carefully document everything then?
How the study was conducted
The study was conducted at Google, which has an interactive code search tool for all of its code. The tool helps their developers find answers to questions related to API usage by allowing them to search within and navigate between files.
The authors used the tool to reach two groups of developers:
Searchers: C++ developers who have API-related questions and hope that the tool can help them answer those questions;
Maintainers: C++ developers who have recently worked on methods that are “popular” amongst searchers/
Searchers who used the tool to navigate from were asked to complete a brief survey about why they chose to do so. Interviews were also held with searchers and maintainers to better understand their perspectives on what should and what shouldn’t be documented.
What discoveries were made
The results are presented separately for each developer type.
What searchers think
Searchers generally expect to find answers to API-related questions in header files: this is often the case with popular, high-quality APIs, which tend to be well-documented. Comments in the header files may include helpful suggestions that document best practices, explain what a method does, how it works, and how it should be called.
Unfortunately, documentation can also be inaccurate, incomplete, incorrect, or simply not exist. That’s why some searchers prefer to read the code instead, which is somewhat guaranteed to provide the correct answers to any technical questions the searcher may have. This works especially well if the code is simple: comments rarely offer any advantage over source code in such cases.
What maintainers think
Maintainers generally believe that documentation should only be added when it’s really necessary. Methods with obvious signatures should not be documented, and searchers should have sufficient background knowledge to understand non-functional aspects of API selection and usage – otherwise they are not part of the API’s target audience.
Comments also shouldn’t be written if there is no intention of maintaining the code, if the code is still actively being evolved or fixed, or when good enough documentation exists for .
Finally, maintainers attempt to preserve the existing comment style when adding new comments to an existing codebase, and emphasise the importance of writing comments that aren’t likely to become outdated.
Documentation rots faster than code, so don’t write too much of it – if you do write a comment, make sure it can stand the test of time
Document only the functionality of a method and the way it should be used, and ignore non-functional aspects
There’s no need to document code that’s self-explanatory, but code that may be hard to understand should be commented
Comments preferably also convey high-level information about things like recommended usage and context of an API (method)