Auto-documentation in Forge

[dkoeplin]

Adds autodocumentation features to Forge, implemented as sphinx as a new backend from Forge DSL specs.

Documentation auto-generated for OptiML

Features

Implemented in preprocessor + backend gen rules

@dsl Preprocessor Annotation Syntax: @dsl as an annotation on DSL spec trait Purpose: Used at the top of a file to tell the preprocessor to look for comment blocks in the DSL spec. Not required, but the preprocessor may miss the first comment if this isn't included in the spec file.

DSL Description Blocks Syntax: Comment block with 3 asterisks

/***
 * DSL description
 ***/

Purpose: Used to populate the introduction page in the generated documentation. Note: Only one is allowed per DSL - if more than one is used, the last one seen by the compiler is used.

Type/Method Description Comment Blocks Syntax: Comment block with 2 asterisks

/**
 * Type/method description
 * @param a description of an unnamed method parameter
 * @param parameterName: method parameter description (ignored for types)
 * @returns description of return of method
 **/

Purpose: Used to describe DSL types, methods, their parameters, and their return value. @param and @returns are optional.

Note: The generator uses signature order to match @param descriptions up with corresponding parameters, and will warn the DSL author if there is a mismatch in number of @param descriptions versus actual method parameters. Parameter name in documentation is chosen in the following order:

1. Name given in method spec, e.g. MethodSignature(... ("value", MInt) ...)
2. Name given in @param, if one exists
3. "Prettified" name (replacing __argX)

NoDoc "Annotation"

Syntax: ```/** @nodoc **/ Purpose: Used to tell the generator that the specified method should not be included in the DSL documentation.

Note:By default, compiler methods and methods starting with "forge_" are not documented.

Notes

Parsing comment blocks only occurs if the preprocessor is not in a method (e.g. implementation/generation rule) scope, so comments in code generation rules and the like should still be safe. Comment blocks beginning with only one asterisk and single line comments are never preprocessed.

Markup is technically supported in comment blocks, but may be a little buggy right now.

Documentation is currently automatically grouped into the following 6 categories:

1. Type classes - self explanatory
2. Data structures - types which have associated data(... ) declarations AND external types which are declared as data structures by adding them to the primitiveStructs list
3. Objects - non-data structure groups which have at least one static method
4. Primitives - staged types which are not data structures
5. Generic methods - operations on type parameters (e.g. asInstanceOf)
6. Operations - operation groups without associated static methods or types, e.g. OptiLA's BasicMath

TODOs:

• Type params currently don't get generated in method signatures.
• Type class evidence constraints are always generated in the more verbose syntax, e.g. (implicit ev1: Arith[T])
• Scala/java types in method signatures use their full names (e.g. "org.joda.time.format.DateTimeFormatter")
• Generic method grouping is a bit broken currently (asInstanceOf is grouped with T:Numeric methods)
• Possible for a primitive type to show up as a static object if it has static methods defined on it (unconfirmed)
• Higher kinded types with multiple type parameters sometimes don't work e.g. (from OptiML's classifier): def logreg(data: Rep[TS[Double,:doc:boolean]]

[asujeeth]

@dkoeplin this is great, thanks! can you open a corresponding PR on hyperdsl that includes these changes so that we can verify Jenkins and merge there as well?