Everett Toews

A common complaint about many open source projects is documentation. Insufficient, incorrect, non-existent, hard to find, and difficult to update are things we typically all hear. There are a lot of different ways to tackle these problems. There’s no silver bullet but one of my favourite tactics is lowering the barriers for absolutely anyone to walk up and contribute documentation.

Lowering the Barriers

Lowering the barriers means:

1. Using a platform conducive to contributions
2. Using ubiquitous documentation editors
3. Using a common and well understood documentation markup language
4. Not requiring your contributors to install and configure tools
5. Providing a preview of the updates to make reviews easier

I built a system for the Apache jclouds community that lowers these barriers. But before getting into the details of exactly how I implemented this, let’s take a look at the process of actually making a contribution.

Ignite Talk at OCSON

At OSCON 2014 I was fortunate enough to be selected to give an Ignite talk on the topic of documentation contribution. Below is a video from that Ignite.

The Process

There are 3 actors in this process:

• The contributor who wants to improve the jclouds’ documentation.
• The reviewer who is most likely a committer on jclouds that will review the changes.
• The doc system responsible for building the staging website and making it available to the contributor and the reviewer.

This is the process they go through:

1. The contributor, wanting to make an improvement, clicks on the Fix This Page link at the bottom of any page on the jclouds website (screenshot).
2. The contributor edits the page while GitHub automatically forks the jclouds-site repo in the background, if necessary (screenshot). Note that the contributor needs a GitHub account and will need to be logged in.
3. The contributor proposes the edits while GitHub automatically create a branch for the commit (screenshot).
4. The contributor creates a pull request (screenshot).
5. The doc system automatically builds the entire jclouds website with the contributor’s edits included and comments on the pull request when done (screenshot).
6. The doc system automatically uploads the staging jclouds website to Rackspace Cloud Files and comments on the pull request with a link to the staging website when done (screenshot).
7. The contributor and the reviewer both click on the link to the staging website so they can both review the exact same rendered changes (screenshot).
8. If the reviewer requests changes, the contributor edits the file(s) within the pull request and proposes those edits in an additional commit. The process goes back to #5 (screenshot).
9. The reviewer rejoices when the changes look good and merges the pull request (screenshot).

While this might seem like a long process, it can actually be very quick. The process could happen in less than 5 minutes, if a simple edit required no additional changes and the reviewer signed off right away. It’s important to note that there is no incidental complexity here. The process is as efficient as it could possibly be with respect to GitHub’s workflow.

The Implementation

Here’s how I implemented this doc system in the jclouds community.

1. A GitHub account is needed to start a pull request
2. Markdown and HTML/CSS with Jekyll
3. Any web browser
4. Jenkins to install and configure the doc build tools so the contributor doesn’t have to
5. Jenkins’ jobs to build the website with Jekyll and to upload the staging jclouds website to Rackspace Cloud Files using jclouds

GitHub

Many people have GitHub accounts already and getting one is trivial. It also gives the jclouds community a good idea about who is making the contribution. The pull request is the primary unit of collaboration on GitHub.

Markdown and HTML/CSS

Simple Markdown documents with HTML/CSS make up the content of the jclouds website documentation. They are combined using Jekyll, a template engine, so that contributors can focus on the content and not the layout of the entire website.

Web Browser

It doesn’t get anymore ubiquitous than the web browser. The Markdown documents can be edited in a textarea on GitHub from a web browser. The web has trained us to fill out online forms and this is no different. You could just as easily go through the process above on your mobile phone.

Jenkins

Jenkins, at the heart of the doc system, is responsible for building the website with Jekyll and uploading the staging jclouds website to Rackspace Cloud Files using jclouds. Installation and configuration of Jenkins itself is outside the scope of this post but it’s relatively easy to get started using it. You need the GitHub Plugin and configure a webhook to fire when a pull request is made to your GitHub repo.

Jenkins Jobs

Jenkins jobs are what get the work done in the jclouds doc system. The 2 jobs are:

1. jclouds-site-pull-requests to build the website with Jekyll
2. jclouds-site-staging to upload the website to Cloud Files with jclouds.

Job: jclouds-site-pull-requests

Relevant settings:

• Project name: jclouds-site-pull-requests
• Source Code Management
  • Repository URL: git://github.com/jclouds/jclouds-site.git
  • Branches to build: origin/master
• Build Triggers
  • Build pull requests to the repository: Checked

Execute shell 1:

More relevant settings:

• Post-build Actions
  • Files to archive: _site/*/
  • Projects to build: jclouds-site-staging

Job: jclouds-site-staging

Relevant settings:

• Project name: jclouds-site-staging
• Source Code Management
  • None
• Build Triggers
  • Build after other projects are built: Checked
  • Project names: jclouds-site-pull-requests
• Build
  • Copy artifacts from another project
    • Project name: jclouds-site-pull-requests
    • Which build: Upstream build that triggered this job
    • Artifacts to copy: _site/*/

Execute shell 1:

Execute shell 2:

comment.py

The jclouds community actually uses CloudBees extensively as the place to host our Jenkins jobs. However, these jobs have been generalized and should run on Jenkins with the GitHub plugin.

Coda

This isn’t just about documentation; it’s about building community through contribution. If you can get a user to make a documentation contribution, that user will be much more engaged in your community. They will have feeling of ownership and pride in a piece of software that they depend on and that they have improved. These are the users who will be with you through thick and thin, and who will find other ways to contribute to your project.

You might not be able to implement this system exactly as I’ve done it in the jclouds community but the important part is finding ways to lower the barriers to contributing documentation. I didn’t build this system in a day. Tackle each of the barriers above one at a time. Eventually you’ll be able to put the pieces together into a system that works for you.