Building Symfony Docs

The documentation of the Symfony project is Open Source and managed using the github.com/symfony/symfony-docs repository. Thousands of people from all over the world have contributed to these docs. It's one of the most popular doc projects of the entire Open Source space.

Symfony Docs use reStructuredText (or RST for short) format to write its contents. Then, we use Sphinx, a documentation building tool, to transform those RST files into the HTML contents served on our website.

This setup has served us well for some years, but it had some serious problems for us. First, Sphinx and most of the tools related to RST are written in Python programming language, which we don't know well. Second, updating and maintaining Sphinx caused many troubles for us, because it's too strict and unforgiving.

That's why around 2019 we started to seriously look for an alternative based on PHP.

Sticking to RST format

An obvious solution would have been to switch Symfony Docs to Markdown, the most popular doc format. This was quickly discarded because of the colossal amount of contents of Symfony Docs (converting all of them to Markdown would take us forever).

Before continuing, let's play a game: can you guess the size (in lines, words and characters) of the entire Symfony Docs? I'll show you the answer at the end of this blog post.

Apart from the contents size, there was another problem. Markdown is great for simple contents, but it's not that great for complex technical docs. RST provides all the features that you'll need when writing complex docs, such as cross references to internal doc sections and programmable "directives" to create custom content blocks. In Markdown it's impossible to do that or you need to use hacks which aren't part of the standard.

Building a RST parser in PHP

Building a doc parser is not a simple feat, much less for a full-featured format like RST. Luckily, Symfony was not the only PHP-related project that needed a RST parser. Doctrine project uses RST for its docs too and also wanted to get rid of Sphinx to build their docs.

That's why some Doctrine people created (and open sourced) the doctrine/rst-parser project based on a previous but incomplete RST parser. It's not a fully-compliant RST parser, but it's super fast, easy to use and covers all our RST needs.

The RST parser problem was now solved. Next, we needed to create a full "doc builder" to replace Sphinx.

Creating a Doc Builder in PHP

Thanks to the flexibility of Doctrine's RST parser, we could build an application on top of it called symfony-tools/doc-builder. In short, this tool finds the RST files, parses them into HTML (including code highlighting), and saves them in JSON files with a certain structure expected by symfony.com.

This doc builder also defines and processes all the custom RST directives used in Symfony Docs. For example, when you browse the Symfony routing article, you can select the format of certain code blocks (annotations, attributes, YAML, XML, etc.) This is not part of the RST standard but, contrary to Markdown, in RST there's a standard way of extending the format.

That's why Symfony Docs use a proprietary .. configuration-block: directive which is processed in the ConfigurationBlockDirective class of our doc builder.

Updating Doc Internals in symfony.com

All the necessary pieces to replace Sphinx/Python with our own PHP Doc Builder were ready. The last step was to update symfony.com code to make use of it. This took us a bit longer than expected because of the "technical debt" accumulated during the past years.

Thankfully we had some PHPUnit tests in place to check that the docs worked as expected (e.g. expected page titles and table-of-contents, doc navigation, etc.) This gave us some confidence before starting this big refactoring.

We ended up deleting most of the existing code related to docs and replaced it with some DTOs created from the JSON files generated by the doc builder. These DTOs contain all the data needed to fully render a page (its table-of-contents items, the locale of the contents, the other Symfony versions in which the page is published, etc.) This helped us remove all the logic from controllers and templates.

Sphinx and Python were now gone. The RST parser and the Doc Builder had been fully integrated in symfony.com. What was next? Redesigning the doc section!

Redesigning the Website

A few months ago we unveiled the new Symfony Bundles Doc section. This was the first symfony.com section to use the new RST parser and Doc Builder. We also created a unique design for it, completely different to the rest of the site.

From the very beginning, this bundles section was a test ground for the Symfony Docs redesign. It allowed us to test the new doc building for real, to perform some experiments and to get feedback from some members of the community. We used all that to build the new Symfony Docs website.

The main ideas behind the new design were:

• Overall design should be simpler, with less "noise"
• There should be more "breathing space" between contents (pages are wider now)
• Readability should be the top priority (larger and better typography)

Here's a comparison between the old (left) and the new (right) index page (click to make it larger):

Here's a screenshot of a typical book page under the new design (click to make it larger):

The new design is full of little details that help improve readability. It's also fully responsive and provides a dark mode alternative. The default mode is the same as your operating system. If you want to set it to light or dark explicitly, use the UI element located in the footer of all pages.

Thanks to All Who Made this Possible

The new Symfony Docs website is a good example of the advantages of Open Source in practice. We leveraged (and contributed to) different projects created and maintained by different organizations. The result is a PHP doc builder which is faster and much easier to maintain than the previous non-PHP builder.

Thanks to all contributors who made this possible:

• List of contributors of doctrine/rst-parser
• List of contributors of symfony-tools/doc-builder

Did you guess the Symfony Docs size?

At the beginning of this article I asked you to try to guess the size of the Symfony Docs contents. Here's the answer: as of October 2021, and according to wc command, the entire Symfony Docs contents across its 25 versions (from 2.0 to 6.0) and including all code examples, comprises 2,712,374 lines, 9,787,338 words and 97,698,319 characters.

To put things in perspective, the "Lord of the Rings" trilogy contains 576,459 words, making Symfony Docs 17 times bigger.