All Articles

Literate Programming in Rust

The idea

It’s no secret that better documentation for code leads to better code quality. When we, as programmers, are forced to explain how code works, we can more easily uncover logical flaws in our designs. It also ensures that those who use, test, extend or replace our code (including our future selves) are not bewildered when trying to understand how things work. This flows naturally from Donald Knuth’s concept of literate programming.

I’ll add a disclaimer that this article is meant to briefly showcase how everyday programming can be influenced by the idea of literate programming. For a technical overview of that topic, visit Knuth’s page, and for a more direct port of WEB/noweb (the canonical literate programming language) to Rust, you’ll want to take a look at the tango project by pnkfelix.

Literate programming

Wikipedia gives us this definition:

Literate programming is a programming paradigm introduced by Donald Knuth in which a program is given as an explanation of the program logic in a natural language, such as English, interspersed with snippets of macros and traditional source code, from which a compilable source code can be generated.

Knuth continues…

The philosophy behind WEB is that an experienced system programmer, who wants to provide the best possible documentation of his or her software products, needs two things simultaneously: a language like TeX for formatting, and a language like C for programming. Neither type of language can provide the best documentation by itself; but when both are appropriately combined, we obtain a system that is much more useful than either language separately.

So now we have the concept (if very naively). Let’s see how we might think about this in the context of the proverbial Rustacean.

A brief tour of doc generation

When writing Rust code, documentation is produced by running the following:

$ cargo doc
# generates html in target/doc

The output is tweakable via flags, such as:

$ cargo doc --open
# opens the generated docs in the default browser
$ cargo doc --no-deps
# does not generate docs for upstream dependencies

Doc comments, written with /// instead of //, support full Markdown syntax. This means we can write something like:

/// # Chapter 1
/// ## My Day
/// Dear diary:
///
/// Today I read a great quote from [Grace Hopper][1]:
/// > "A ship in port is safe, but that's not what ships are built for."
///
/// [1]: https://en.wikipedia.org/wiki/Grace_Hopper

and it will render as such: Markdown as rendered by rustdoc

Neat! We can embed links, images, tables and have rich formatting right in the documentation. For all intents and purposes, our documentation is a web page. The immutable.rs docs are a shining example of just how much detail can be expressed in the format.

As you may know, Markdown automatically generates appropriate syntax highlighting, that is:

```python
def hello_world() -> str:
    print("Hello, world!")
```

is rendered as

def hello_world() -> str:
    print("Hello, world!")

In Rust documentation, this capability is extended: If no language is specified in the opening fence, cargo will interpret the code as .rs source, and will automatically compile and run anything within the fences whenever cargo test is run. Each fenced block is compiled into its own implicit binary instance.

This is an excellent constraint; it forces the developer to ensure that any code within the block is using the library exactly as an external user would.

Let us see how this looks in practice.

Say that we’re developing a crate called “maths”, and our lib.rs has a simple incrementer function for the i32 type

pub fn inc_i32(n: i32) -> i32{
  n + 1
}

Let’s first add some descriptive content

/// increments the given integer by one.
pub fn inc_i32(n: i32) -> i32{
  n + 1
}

and then test our incrementer right in the documentation

/// increments the given integer by one.
/// # Example
/// ```
/// let x = 9;
/// let answer = inc_i32(x);
/// assert_eq!(answer, 10);
/// ```
pub fn inc_i32(n: i32) -> i32{
  n + 1
}

We run cargo test and… Oh no! It failed

running 1 test
test src/lib.rs - inc_i32 (line 3) ... FAILED

failures:

---- src/lib.rs - inc_i32 (line 3) stdout ----
        error[E0425]: cannot find function `inc_i32` in this scope
 --> src/lib.rs:5:14
  |
4 | let answer = inc_i32(x);
  |              ^^^^^^^ not found in this scope

🤔 Ah, remember, this is running in a sandbox. We have to import our library for it to work.

/// increments the given integer by one.
/// # Example
/// ```
/// use maths::inc_i32;
///
/// let x = 9;
/// let answer = inc_i32(x);
/// assert_eq!(answer, 10);
/// ```
pub fn inc_i32(n: i32) -> i32{
  n + 1
}

The tests now pass.

   Doc-tests maths

running 1 test
test src/lib.rs - inc_i32 (line 3) ... ok

and the example renders as expected A rendered doctest example

This is fine for such a trivial function, but in a more realistic piece of code, more setup would likely be needed before the test will pass. This could clutter the generated documentation and make the examples hard to understand. Within Rust code fences in doc comments, adding a # character before a line of source will cause it to be elided when rendered.

An example:

/// increments the given integer by one.
/// # Example
/// ```
/// # use maths::inc_i32;
/// let x = 9;
/// let answer = inc_i32(x);
/// assert_eq!(answer, 10);
/// ```
pub fn inc_i32(n: i32) -> i32{
  n + 1
}

and we see in the rendered output, the use declaration is neatly elided: An example with 'use' elided

Conclusion

Let’s go back to literate programming, and emphasize one line in particular:

…needs two things simultaneously: a language like TeX for formatting, and a language like C for programming.

I would argue that this is exactly what we’ve got in the Rust doc system! Markdown for formatting, and Rust for programming. The fact that the doctest is sandboxed in such a way that we are forced into scoping our public API is orthogonal to this, but a huge bonus. This means once we think we have expressed a literate idea, we still need to prove it to an unforgiving main.rs that knows nothing about our code.

Discussion

What do you think of the “literacy” of cargo + rustdoc? What tools do you use in your programming ecosystem to accomplish the same goals? What is the experience like in doxygen, sphinx, hackage, etc.? Drop a comment below 👇

Published 9 Aug 2018

👨🏻‍💻 :: (coffee, beer, music) -> ideas
Damien on Twitter