Auto-translation of NIST controls to Controls Markdown Language for human editing and maintenance

[rafael5]

User Story:

As an OSCAL {stakeholder}, I ... want to be able to read and edit the controls in human readable form on the Github respository using a Controls Markdown Language.

This is Controls Markdown Language would be similar to (and preferably compatible with ) the OpenControls YAML representation, but represent the OSCAL model. This would allow all controls to be easily readable and maintainable by humans with only simple text editor.

Goals:

Maintain all the controls in the Github repository using human-readable Controls Markdown Language. This markdown format must be interchangeable (by automated scripts) to the current OSCAL XML and JSON, but in human-readable and editable form.

Dependencies:

1. The Controls markdown language needs to be compatible with and fully represent all nuances of the published NIST controls with no loss of information content or context.
2. The Controls Markdown Language needs to be compatible and feed into HTML/PDF document production pipelines (either asciidoc or reStructuredText, leveraging either Pandoc or Sphinx document production libraries respectively).

Acceptance Criteria

The Controls Markdown Language needs to pass

1. Automated validation against existing OSCAL XML/JSON formats to preserve all content
2. Automated validation testing against the original NIST published control
3. Automated document production (HTML/PDF) using either reST/Sphinx or asciidoc/Pandoc.

[anweiss]

@rafael5 can you elaborate on this a bit? what advantages would such a language provide over YAML? are you referring to a superset of Markdown? And by "controls in the GitHub repository" are you referring to the OSCAL-formatted 800-53 catalog example?

[trevor-vaughan]

I'd like to link this back over to https://github.com/usnistgov/OSCAL/issues/216#issuecomment-425208258 because my concerns are the same.

[rafael5]

1. re: what advantages would such a language provide over YAML? I am not concerned with the actual implementation language so long as it is easily human readable, editable, and maintainable. OpenControls approach (using YAML) was a simple effective approach. However, it may not be robust enough as a Markup language for controls if it cannot to logic bindings, inheritance, relations, and indexes. So it cannot be primitive like Github flavored markdown, which has no capability to index or create a table of contents. An analogy would be reStructured text or asciidoc markdown, which are much more powerful and can generate whole books, table of contents, indices, etc.
2. re: are you referring to a superset of Markdown. In a word, yes. However I am not even saying we have to start with Github Markdown (There are many different flavors of Markdown, which limits is utility as a standard - i.e. it is a nonstandard standard). Per #1, we would require a flavor of Markdown that can handle indexing, relations, hierarchies, interitance, and other logic inherent to controls managment (such as mapping, reconciliation and automation).
3. re: " controls in the Github respository": I mean NIST 800-53 controls represented in Controls Markup Language (a yet-to-be-defined specification) hosted on Github, and editable using Github text edting tools. This Controls Markup Language would be backwards compatible with the XML/JSON of OSCAL (via automated translation)

Does that help?

[anweiss]

Thanks @rafael5. Indeed this helps to clarify. I also forgot about @trevor-vaughan’s comments from a while back re. his similar ask. Would love to hear from @wendellpiez on this as well.

I’m certainly not against this and see the value in having such a language. We’d have to keep in mind that this would add a bit of extra overhead for tool vendors in that they’ll now have to implement a parser for such a markdown language. I also imagine that the language would have to provide some sort of abstraction over the breadth of options provided by the underlying OSCAL data models.

On the other hand, one has to ask to what extent folks are going to be hand-writing OSCAL, regardless of the existence of such a markdown language. At the end of the day, the actual compilation of the data which becomes OSCAL’ized is where folks will likely spend cycles. So whatever can be done to make that process easier (via tools, UIs, markdown languages, etc) I think is super important.

[anweiss]

@rafael5 would also like to see some prototypes of what you have in mind so we have something a bit more tangible to look at.

[david-waltermire]

We have considered producing a YAML-based version of the OSCAL models. This would be another output format of our metaschema-based production process. We could use this to autogenerate documentation and conversion scripts to and from OSCAL YAML from the XML and JSON formats in a similar way to how we autogenerate the XML and JSON schema, related documentation, and XML <> JSON conversions scripts.

[guyzyl]

@david-waltermire-nist would a basic OSCAL YAML to markdown script suffice here? Or is a 2-way conversion needed.

[iMichaela]

@guyzyl - you might want to see the IBM work here: https://ibm.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring/... The description reads: "In summary, the catalog tools allow conversion of a Catalog to markdown for editing - and back again to a Catalog. The profile tools similarly convert a Profile's resolved profile catalog to markdown and allow conversion to a new Profile with modified additions that get applied in resolving the profile catalog. Finally, the ssp tools allow the addition of implementation prose to a resolved profile catalog, then combine that prose with the Catalog into an OSCAL System Security Plan."

[chaffin]

Perhaps a legacy documentation system such as https://www.gnu.org/software/groff/manual/groff.html#groff-Capabilities would suffice instead of implementing another Markup/Markdown language. The use of macros allow embedded formatting commands as primitive markup and provides many capabilities you have outlined.