Improve the presentation of grammar elements

[krilnon]

Right now, the grammar in Language Reference is implemented using plain code listings. The shipping book includes a custom presentation for the grammar, including cross-references between different grammar elements and a page that hosts the generated summary of the grammar.

[amartini51]

The current book's approach to styling the grammar tries to be as easy to read as possible — even when that makes it a little more difficult to author, compared to a plain-text grammar like BNF. Some enhancements to DocC that we can consider to improve the experience of readers looking at the grammar:

• The name of syntactic categories is set in roman type, and is a live link to its definition. The live link makes it easy to follow a series of production rules, one step at a time. We don't currently have the ability to link to somewhere on the page other than a heading.

• Literals are styled as bold monospace font, to make them easy to distinguish even for single characters, where the distinction between monospace and roman is less visually apparent. We can currently mark up text as monospace or bold, but not both at the same time.

• Optionality is marked by "opt" in subscript. We can't currently mark up subscripts or superscripts.

I've also noticed that when authors of Swift Evolution proposals are writing about impact on the grammar, they're usually writing in GitHub flavored markdown which also doesn't support subscripts, so they fall back on various other spellings like "-opt" or ?.

[amartini51]

Filed https://github.com/apple/swift-book/issues/48 to track navigation separately as a "parity" issue.

One approach we could take using the markup currently available in DocC is to use note boxes.

• Use a note title like "Grammar of an expression"
• Write definition a new paragraph, so it appears on its own line
• Use a Unicode arrow to separate the two sides of the rule
• Use code voice for literal tokens (bold code voice isn't supported)
• Use a suffix "?" to mark optionality (subscript isn't supported)
• Use plain (aka roman) font for English descriptions of rules

Issues:

Using "?" for optionality has a couple issues. It can visually look like a literal token. And when it comes after a literal token, it can be hard to tell which part is in code voice. Similar approaches like using -OPT or (opt) to mark optionality have these issues too. We can improve this by adding support for subscripts to DocC

Using a paragraph break after every rule loses the ability to group rules together. For example, the grammar for integer literals has "paragraphs" for binary, decimal, octal, and hexadecimal literals. Removing that grouping will make the grammar harder to read. We can improve this by adding support for hard breaks to DocC.

[amartini51]

Correction: DocC does support bold code voice, which I think makes using italic ? to mark optionality visually distinctive enough. There aren't any grammar productions that include an optional ? token, so we wouldn't have ? followed by ? anywhere currently.

[amartini51]

I've pitched this approach for grammar on the Swift forums.