Document Like You Code

Written by Rod Lambert, Software Engineer at Stout Systems

Documentation types

A well-run organization that develops software, hardware, or services, requires a large amount of documentation to ensure the products and services meet their objectives and timelines of the organization. Creating the documentation, tracking changes, and informing the stakeholders is often difficult and poorly executed.

Here is a sampling of the various document types you may need to track:

Product

• Product requirement document
• Top level design document
• API documentation
• System level documentation
• UI documentation
• User documentation
• Certification agency documents
• etc...

Process

• Product proposals
• Coding standards
• Strategic plans
• Development operation documentation
• etc ...

Changes to any document usually requires notification to and input from various groups, thus illustrating the complexity of the documentation process. Once changes are made there is often not a process in place to ensure that all parties affected by the change are notified.

Version control is essential for documentation and source code

If your organization creates software, then you likely have a version control process in place. There is also a good chance that your development process gets more benefit than just a collection of different versions. The formal process of branching, creating a pull request, and code review disseminates information to your developers and helps them understand how your changes will affect other areas of the project.

Use a version control system for all of your documentation

When your organization starts to treat its documentation like it is source code, then the benefits that the version control process provides can start to automate your product design and maintenance. Documentation crosses into more areas of most organizations than source code does and process will provide surprising linkages between groups.

Imagine your organization is developing a new product. During discussions with a potential customer someone in the meeting discovers that a change needs to be made to meet regulatory requirements for the product. Since all levels of the design document are under version control and pull requests need to be reviewed by all group leaders, when changes are made to meet this new requirement all affected groups will be part of the document change process. Like any pull request, comments about the impact and extent of other changes are discussed. Tasks to modify any other documents, electrical design, firmware, software, testing, and timelines can all be created.

Something very special can happen

Version control systems are powerful tools for developing complex systems but are typically only used by the software group. When an organization moves all of its documentation into version control it crosses all groups within the organization. The organization now has a process in place that allows changes made by one group to percolate throughout the organization to all other groups. Now, all groups can be informed or participate in the process, as needed.

As an example, imagine that the sales group is working with the frontend group to improve the user experience on a section of the website that shows the status of hardware that collects data from a network of remote sensors. The working group of sales people, frontend developers, and backend developers agrees on adding a button to the UI that would allow the client to verify that any specific remote sensor can be pinged to verify that the connection to the device can be made. When they do the pull request to change the UI design document, the leader for the group that is responsible for communications with the remote sensors immediately realizes that there is a problem with the proposed change. The issue is that the server the client is interacting with cannot directly connect to most devices because it cannot access the VPN that the devices communicate on for security reasons. The backend server would need to identify the server that can actually test the network connection, ask that server to schedule a job to test the connection to the sensor, and then communicate to the backend the result of the test. The complete transaction is likely to take far too long to be displayed directly as the button is clicked by the user. At this point, additional people can be added to the pull request or hold an expanded meeting to propose solutions that can achieve the desired result.

Figure 1: How the working group expected the feature to work

Traditionally organizations would attempt to avoid the example above by having more people in the working group meetings. Adding more people to the working group has the detrimental effect of consuming resources. It also reduces the effectiveness of the meeting. The worst part is that the actual agreed upon changes are not always well understood after the meeting.

Figure 2: How the feature would actual have to work

How do we effectively version control our documents

If your organization is already using version control for source code then convincing people to use version control for the organizations documents has a chance of working.

Version control for source code works well because the text based nature of the code allow changes to be localized and easily compared. Therefor the first requirement is that the documents be pure text and not contain any opaque binary parts.

To meet this pure text requirement one might immediately consider using What You See Is What You Get (WYSIWYG) document editing software like Microsoft Word or LibreOffice Writer and then only creating text based output file types. This is where the requirement that changes need to be localized will come back to absolutely destroy any hope of using version control on them. Document editing software creates documents of a specified format but for the most part the application is free to arrange the output file as it sees fit, as long as it produces a view of the document that the creator intended. For example, changing a single word from normal to bold weight can result in the entire xml document being reorganized! You need to avoid any approach to creating documents that can result in non-localized changes. Saying it another way, if you want to change a word from normal to bold weight that change should be easily understood in the difference between versions of the file.

Many of your organizations technical documents likely contain rich graphical content. Often that content is brought into the document as a binary image or linked in via the producing application. These are also documents in their own right and should also be under version control. For example, many product design documents contain flow charts that describe aspects of the products functionality. It is common to create the flowchart in an application that specializes in flowcharts and then save the flowchart as an image that gets embedded in the product document. As the product is developed those flowcharts change but there is no clear and easy process to have the binary images inside of a binary file get updated as the product evolves.

Remove every binary file from your documents that you can because they are documents that cant be under proper version control. There are limits to how far you can take this approach. For example, product photos are going to remain and you may have output from other applications that you will need to be included as images, but the text-only approach can go way beyond what most people would believe is possible.

LATEX is the best choice for documentation

LATEX is a type-setting system. You write your document content and formatting in plain text and then that text document is compiled into the output document. Since all of the content and formatting are just text, a LATEX document is an ideal candidate for version control. The LATEX document is the source code that instructs the LATEX compiler what the output document should look like.

LATEX is free and supported on all major platforms (Windows, Mac OS, Linux) and there are even online solutions. There are standalone editors that specialize in LATEX editing but the documents can be written in a simple text editor. One popular choice is to use Visual Studio Code which has plugins that can integrate the writing of the LATEX document and the generation of the output document.

LATEX uses packages to extend its capabilities

Much like a programming language, you can include packages into your document that provide additional capabilities. Some packages are geared toward specific formatting options and others bring in the ability to generate plots or draw graphics using only text.

Imagine including a flowchart in your documentation in a conventional WYSIWYG editor. You would typically create the flow chart in another application and then include it into the working document. Most likely you would use an image of the flowchart or link to another document from the application that created the flowchart. In either case you cannot make changes to the flowchart in the same manner that you would make changes to source code because the document that you are modifying is not text based and the flowchart that it includes is not text based. In LATEX you can include that flowchart into the document as text.

Adding graphics to LATEX documents

Most documentation of technical material will require more than just text to most effectively communicate the information. LATEX is capable of including complex mathematics, data plots, and drawings directly into the document in plain text data and commands. When the document is compiled the compiler generates vector graphics based on those commands. The list of packages and document templates that help in this process is well beyond the scope of this article but it is worth mentioning the TikZ package.

TikZ is markdown language that produces vector graphics. Once you include the package into your LATEX document you can define a wide variety of images using only text. The flowchart in Figure 3 was created from approximately 20 lines of text.

Include files into a document

LATEX files can include other files with a process similar to how you would include a header file in C++ code. For example, this document was generated including another document called FlowChartStyles.tex that defined the style of flowchart elements. This is a fundamental change in how documents are styled because you can include the style document in all of your organizations documents. Then when you make a change to one of the styles, every document that uses the style will update when it is compiled. The changes also extend into the graphics.

For example, if your organization uses a style file to define the flowchart element styles, then every document will have consistent looking flowcharts. Now lets say that the decision is made to change the fill color for decision blocks. That change will be propagated through every flowchart in every document that uses the style.

Figure 3: Simple flowchart example

LATEX can execute commands

If the correct packages are included, your documents can call external scripts, use variables, and include logical statements that control the output. The possibilities that these tools open up are extraordinary.

Consider an integrated circuit manufacturer that produces multiple ICs that use the I2C interface. It would be expected that there would be multiple timing diagrams of the communications process. Examples would have different chip addresses and different data, but one could create a parameter based I2C timing diagram that can be included in multiple locations with the appropriate parameters being set for each desired figure.

What is the downside of using LATEX

LATEX is a complex language the will be new to most people. Their experience gained using WYSIWYG systems will not help them much in the type setting world of LATEX. Also they may find the edit, compile, and view workflow to be strange.

The version control process will also be new and surely confusing to those who dont have experience using such systems. Even with graphical application front ends there is still a fairly steep learning curve.

Unless you convert existing documents there will be a need for two document workflows, one for the new version controlled documents and one for the older uncontrolled documents.

Lead by example

This article was created entirely from text in LATEX. The complete source files can be found at https://www.bitbucket.org/stoutsystems/DevelopmentDocumentationArticle.git.

This is a technical/business article catered to developers, hiring/project managers, and other technical staff looking to improve their skills. Sign up to receive our articles in your email inbox.

If you're looking for a job in the tech industry, visit our job board to see if you qualify for some of our positions. If you're looking to hire technical talent for your company, please contact us.

Stout Systems is the software consulting and staffing company Fueled by the Most Powerful Technology Available: Human Intelligence. We were founded in 1993 and are based in Ann Arbor, Michigan. We have clients across the U.S. in domains including engineering, scientific, manufacturing, education, marketing, entertainment, small business and robotics. We provide expert level software, Web and embedded systems development consulting and staffing services along with direct-hire technical recruiting and placements.