10 Documentation
If you look at the documentation of any reasonably maintained package, you will notice that, among other things, it covers the application programming interfaces, or APIs. That is to say, it provides the full description of the classes and functions that the package exposes—not only a top-level summary of how they are intended to be used, but also all the gory details of the arguments and return values of functions and class methods.
Take the documentation of our favorite function scipy.curve_fit(), for instance; you have, in order:
- the full signature of the function;
- a short, top-level description;
- a detailed description of all the arguments that can be passed to the function;
- a detailed description of the return values;
- a detailed description of the exceptions that can be raised;
- additional notes and reference.
Lots of stuff, eh?
Now, one thing that you might start wondering is: how do people keep the code and the documentation in synch? Do they physically live in separate files? And: if I ever change the signature of my function in the code, how do I make sure I remember to update the documentation? Because if the documentation starts diverging away from the underlying code, that is pretty bad—wrong documentation is definitely worst that no documentation at all.
The main message in this chapter is: documentation should live as close as possible to the code is describe—possibly in the very same file—and the documentation process should be automated wherever it makes sense. Use one single source of truth and avoid copy and paste at all costs.
Simple exercise: click on this link to the scipy.curve_fit() documentation and take a good look. Next to the signature of the function you will find another link to the source code for the same function. Open the latter in another tab and switch back and forth between the two. Anything that captures your attention?
10.1 Sphinx
Different languages have different toolchains of choice, when it comes to documentation. For Python, everybody uses Sphinx. And, in this context, everybody really means everybody: Python, numpy, scipy, matplotlib.
When you think about documenting a software project, there are actually to equally important parts to it, namely:
- descriptive text explaining the purpose and usage of the project (this might include, among other things, a user guide, examples, tutorials and so on and so forth);
- technical documentation (i.e., the APIs), describing the inner workings of the code.
Just to put things in context, the reference page for the optimize module in SciPy is a good example of the first, while the documentation of the curve_fit() function, that we now know by heart, belongs to the second category.
Sphinx handles both things equally well—the first is expressed in the form of a series of .rst markup files, while the second is extracted from the docstrings in the Python code. The whole thing is driven via a python configuration file that you can create with sphinx-quickstart utility.
10.1.1 General documentation
Descriptive documentation goes into dedicated .rst markup files. (As a matter of fact you will need at least a index.rst file that acts as the main entry point for the whole thing. It goes without saying, you can have as many .rst files as you see fit.)
There is nothing particular to say here, except for the fact that Sphinx uses reStructuredText as the particular markup language. The documentation will get you started with the basic syntax.
Sphinx come with an extensive set of built-in extension that do lots of interesting and useful things. At the very least you will want to use
apidoc(generate API documentation from Python packages);autodoc(include documentation from docstrings);napoleon(support for the NumPy and google style docstring).
And of course there are also contributed extensions doing even more amazing thing. You are definitely covered.
10.1.2 Documenting the interfaces
And now about documenting the interfaces. This is typically handled through proper docstrings—properly formatted pieces of text delimited by triple quotes that accompany classes and functions.
We are not going into all the gory details, but basically if instead of coding your functions like this:
def square(x):
return x**2.you do something along the lines of
import numpy as np
from numpy.typing import ArrayLike, NDArray
def square(x: ArrayLike) -> float | NDArray:
"""Return the square of a number or an array.
By virtue of the ``2.`` rather than ``2`` in the exponent, the
result is always floating point, even when the input is integer.
Parameters
----------
x : array_like
Value or values to square.
Returns
-------
array_like
Squared value or values.
Examples
--------
>>> square(2)
4.0
>>> square(np.array([1, 2, 3]))
array([1., 4., 9.])
"""
return x**2.that will go a long way toward a good documentation for your package. You can dive into the Sphinx documentation and find your way have it pick up your docstrings.
Note this is using the NumPy style docstrings. Alternative possibilities include the google style and the plain Sphinx style.
10.1.3 Compiling the documentation
Once you have written the source for the documentation all you need to do is compile. If you have run sphinx-quickstart to start with, you should have a Makefile, along with a make.bat Windows batch file with all the proper targets. You should be able to compile the static html, e.g., by just doing
cd docs
make html10.2 Making the documentation available.
Final step: compiling the documentation locally, per se, is not very useful—if not while you are writing it. In terms of your users what you care most is for them to be able to see it.
Nowadays the standard way to make the documentation available is through a GitHub action. You might have one, e.g., that compiles and deploys the html documentation each time you tag your package.
10.3 A concrete example
And now, if you want to understand how all these things play out in practice, go ahead and look (again) at metarep—make sure to hit the link to the documentation.
There is a lot of overlap with this chapter, but the two really serve different purposes, and together they will hopefully get you up and running.