Support for "Rules" in OSCAL Models

[aj-stein-nist]

User Story:

As an OSCAL tool developer, in order to ensure my software can document testing requirements that an information system must implement as one part of cumulative control implementation requirements, I would like enhancement to the OSCAL models to more explicitly define the concept of a rule as a first-class citizen. Modifications and new additions to OSCAL to tool developers to build software for users to give specific criteria to test for a specific kind of implementation implied by control requirements, and have such criteria expressed in OSCAL.

Goals:

• [x] Define notional use cases for the rule, operationally, then in terms of OSCAL mechanics
• [x] Design supporting structure and insertion points for TBD rule assembly
• [x] #1339
• [ ] #1391
• [ ] #1364
• [ ] Finalize the rule design (in the story above)
• [ ] Update the framing document content into Contents page and other relevant pages as needed
• [x] Analyze use cases and determine where rule assembly should and should not be used in relevant OSCAL models
• [x] Obtain NIST team and community buy-in for design that supports continuum of automated to manual rules

Dependencies:

N/A

Acceptance Criteria

• [ ] All OSCAL website and readme documentation affected by the changes in this issue have been updated. Changes to the OSCAL website can be made in the docs/content directory of your branch.
• [ ] A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• [ ] The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

[aj-stein-nist]

While reviewing the backlog today, usnistgov/OSCAL#821 seems related, but specifically focused on component-definitions. This story work is very similarly aligned, but perhaps greater in scope. User feedback this week indicates it might not be ideal to have such checks only or exclusive in a component, but more modeling work is needed to determine that.

[aj-stein-nist]

First things we ought to tackle:

• How do rules fit into each of the OSCAL models conceptually? (Does the concept of "rules" make sense in a catalog's model and document instance, for example? It seemingly makes sense at first glance in component or system-security-plan, right? Analyze this for each and every model.)
• How we will we connect/reference to the check for the rule (from usnistgov/OSCAL#1059)?
• For third parties, how do we allow them to include checks for a model document instance? They might not "own" the document instance of a model (e.g. a SSP, a component) as its original author? They have to shim it in even if they are not the original author(s).

[aj-stein-nist]

Discussed more with Dave today. Will begin by sketching out the "do rules fit into each of the OSCAL models conceptually?" question as well as gathering consensus on what the the relevant data elements are.

[aj-stein-nist]

@david-waltermire-nist, I know we will have another backlog refinement in the near future, but you had asked if I am ready to pull this one into Sprint 49 if you and team are ready.

[aj-stein-nist]

Notes from our last touch-base with interest community members.

My current action item: build information model outline with recommended changes to include the concepts of rules.

[aj-stein-nist]

NOTE: This draft is obsolete, please see the updated draft spec below.

~Proposed Information Model for rule~

Moving forward, we might suppose a rule can look like this:

• rule [0 to ∞]
  • uuid [1]
  • title [1]
  • description [1]
  • origin [0 to ∞]
  • subject [0 to ∞]
  • props [0 to ∞]
  • link [0 to 1]
    • href
    • rel:
      • satisfies-partially
      • satisfies-completely
      • recommended-with
      • required-with
      • conflicts-with

A rule will be a summary explanation of mechanisms to evaluate and/or test amount of security and/or privacy information in OSCAL, including linkages to relevant controls. I specifically do not call out where rules should be inside a document instance of an OSCAL model yet because that is a separate discussed below.

Because a rule is an explicit mechanism to test or evaluate if a system satisfies a given statement, there are some obvious parts of the information model.

• uuid: uniquely identify the rule in its embedded OSCAL model document instance
• title: a summary title for the rule in question
• description: the brief explanation of the test
• origin: where the rule information originated from, be it a person or an automated assessment tool. We expect there are fully automated, semi-automated, and manual evaluation methods. origin, as it is used now in the assessment-results model
• subject: the subject of assessment, be it a person, process, or system component (service, server, et cetera)
• props: One or more properties with variables and their values to be mapped as inputs to a tool, if the origin is an automated assessment tool.
• link: this linkage is intended to function in two ways: 1) by way of specific @rel values (satisfies-partially; satisfies-completely) denotes if it partially or completely satisfies requirements elsewhere in a OSCAL document instance (e.g. the statement of an implemented-requirement of a control); 2) denotes it's relation to other adjacent rules by way of specific @rel values for the link (recommended-with; required-with; conflicts-with). This can be used in combination to allow specific rules to be annotated in a way for sufficient aggregation of what OSCAL document instance data they satisfy.

[david-waltermire]

A few questions:

• Given the above, if multiple rules are used to "satisfies-partially", how do you define the set of rules that "satisfies-completely"?
• How would you point to the rule or check implementation?

[aj-stein-nist]

A few questions:

* Given the above, if multiple rules are used to "satisfies-partially", how do you define the set of rules that "satisfies-completely"?

I think it will be up operator expertise, unfortunately. I was thinking about this a lot yesterday, and I have no good ideas on how we can determine that without some really complex semantics not in place yet.

But back to the actual question, you could use link/@href to point the implemented-requirement/statement (in the case of a component-definition for example) with a link/@rel bound to satisfies-completely and then set additional link/@hrefs for with the UUID fragements pointing to others independent rules that support that same implemented-requirement/statement and the link/@rel would be set to required-with. I know this leans heavily on link but I didn't know if we wanted new constructs here and additional complexity.

* How would you point to the rule or check implementation?

I think I need to flesh out an example, but that would be the origin construct. Is this a good reuse of such a construct or a bad one?

[iMichaela]

@aj-stein-nist and @david-waltermire-nist Would a numeric value associated with the rel for the satisfaction (aka percentage or score) make sense? In this way, an additive operation can be applied to identify the cumulative effect of partial satisfaction and aggregation towards satisfies-completely. The numeric values would be inputs from authoritative entities that can be identified through their ID, since it would be nice to know who decided this test satisfies 30% and not 80% Also, a field that would indicate this rule is part of a superset of rules what is the uuid of that superset might be nice.

[aj-stein-nist]

@aj-stein-nist and @david-waltermire-nist Would a numeric value associated with the rel for the satisfaction (aka percentage or score) make sense?

Personally, I think it would be hard to measure and quantify, your question makes me believe maybe the satisfies- rel attributes need more thought. I meant it be qualitative and intended for expert human insight, but as I write this I realize that also might be too bold.

In this way, an additive operation can be applied to identify the cumulative effect of partial satisfaction and aggregation towards satisfies-completely.

I like the idea in principle but I have no idea how we could implement this in practice. Do you have any ideas around this? I too will reflect on it.

The numeric values would be inputs from authoritative entities that can be identified through their ID, since it would be nice to know who decided this test satisfies 30% and not 80% Also, a field that would indicate this rule is part of a superset of rules what is the uuid of that superset might be nice.

My intent with link/@rel="required-with" or link/@rel="recommended-with" was to allow for that through a tagging approach. In most instances, I find this more flexible. Are there downsides to this approach?

[iMichaela]

@aj-stein-nist I agree it allows for more flexibility, but I do not see a mechanism for identifying the completness or coverage of the superset of rules. A superset might still provide only partial coverage. Through the aggregation, the superset comprised of satisfies-partially rules could achieve satisfies-completely or still be satisfies-partially. Who makes the decission (e.g. role-id) and based on what evaluation mechanism to support consistent reproduceability? I am not saying a scoring or any numeric value will not factor in subjectivism, but at least it can provide a more accurate representation of the categorization (partial vs complete). Something a tool could parse, process (calculate), keep track of, and report. I am thinking one might want to include rules to carry on evaluation of the technology, without any level of testing or proof of the control sadisfaction. This scenario will provide 0% satisfaction for 800-53 control per 800-53A, but might support vulnerability scanning or alignment with CIS benchmark.

[wendellpiez]

Personally I think numbers are not a good way to represent such a value space. It is not even one-dimensional. Also, they can't reliably or meaningfully be treated as numbers (does satisfies="50" + satisfies "60" = satisfies "110"?) Simple labels are better in that they do not promise what they cannot deliver. If systems have actual ranked/numeric values in a taxonomy they wish to label things with, the model offers prop.

Noting however that the discussion on how to characterize linking semantics is a little different from the discussion on how to describe rules. The problem of describing the rules vs characterizing the relations between rules. For example, we have silently assumed the target of a link from a rule is another rule. But what if we want a link to point to a control or a statement inside a control, for reference (how do we do that)?

The discussion about the meaning of values of @rel makes sense only for links that are more tightly constrained than links in general (that is, links from one rule to another rule).

[iMichaela]

But what if we want a link to point to a control or a statement inside a control, for reference (how do we do that)?

@wendellpiez - Per our initial discussion, rules are associated with controls or statements of controls. I cannot picture a scenario when such a cyclic approach would be necessary (when a rule associated with a control or statement of control needs to link to another control). Do you have an example in mind?

does satisfies="50" + satisfies "60" = satisfies "110"?

It depends on the numeric value :). If it is 50% added to 60% you can only get 100%. If it is a different numeric value (a score) with a range of 0-200, then 50+60=110. If normalization will be necessary, then that can be done with a simple mathematical operation on the range. Do you have another approach that would support automation for aggregation and quantification of satisfaction degree, something GRC tools can implement? How will a bunch of satisfies-partially will be quantified? How will one assessor provide his perspective on satisfaction to another team member or system owner? I am open to better quantification solutions.

[aj-stein-nist]

I think both of you are making good points, @iMichaela and @wendellpiez, and I will likely adjust my draft recommendation to say the link/@rel can only be used to talk about possible relationships between independent rules (e.g. link/@rel="required-with" or link/@rel="recommended-with" or the others) and remove the idea of satisfaction markings we are discussing for this very reason. There is too much ambiguity.

I thought it might add value, for very specific assertions at the statement level, for a system owner or developer (not an assessor) to make very clear what they are and are not claiming about coverage of something. In addition to customer , I thought there might be value for very specific details of an implementation for such personas to point out "passing this rule check covers this specific detail of a statement on an implemented requirement for control AB-1 in its entirety" versus "for AB-1, passing this rule check is a good start, but we are not claiming that is enough and you have more work to do on your own (whether that is something else with the sub-system I provide or someone else, that is for us to discern somewhere else)."

I do not think there will be satisfactory answers to these questions at this point and it will get more difficult moving forward.

[aj-stein-nist]

I have continued trying to formulate examples, and it seems trying to reuse the origin and subject assemblies from elsewhere might not work so well, structurally. That, in addition to the feedback above, probably warrants an update in the design of the rule object proposed above.

[aj-stein-nist]

Trying to work backwards and come up with some examples now.

• Example of linking an automated OpenSCAP rule test to a statement of an implemented requirement of a control for an OpenShift Container Platform component-definition
• A complex listing of one or more interlinked rules supporting one or more statements of an implemented requirement of a control for an OpenShift Container Platform component-definition
• A representation of semi-automated rule completed by human interpretation of a semi-automated result
• A manual rule completed by human interpretation without immediately viable method for automation

[aj-stein-nist]

NOTE: This draft is obsolete, please see the updated draft spec below.

~Proposed Information Model for rule~

Moving forward, we might suppose a rule can look like this:

• rule [0 to ∞]
  • uuid [1]
  • name [0 to 1]
    • title [1]
    • description [1]
    • condition [0 to 1]
    • condition-evaluator [0 to ∞]
      • uuid [1]
      • name [0 to 1]
      • type [0 to 1]
        • prop [0 to ∞]
        • link [0 to ∞]
          • href
          • rel: dependency
    • prop [0 to ∞]
      • name: supports

A rule will be a summary explanation of mechanisms to evaluate and/or test amount of security and/or privacy information in OSCAL, including linkages to relevant controls. I specifically do not call out where rules should be inside a document instance of an OSCAL model yet because that is a separate discussed below.

Because a rule is an explicit mechanism to test or evaluate if a system satisfies a given statement, there are some obvious parts of the information model.

• uuid: uniquely identify the rule in its embedded OSCAL model document instance
• title: a summary title for the rule in question
• description: the brief explanation of the test
• condition: the fact used for evaluation of the confirmation or negation of the rule during a check
• condition-evaluator: where the rule information originated from, be it a person or an automated assessment tool. We expect there are fully automated, semi-automated, and manual evaluation methods.
• condition-evaluator/@prop: One or more properties with variables and their values to be mapped as inputs to a tool, if the origin is an automated assessment tool.
• condition-evaluator/@link: One or more links to a back-matter/resource with @rel="dependency" to link to copies of utilities and supporting configuration files (for cases of automated rule evaluation).
• prop: Additional properties for arguments to the rule. At this time, one explicit prop designed for use within the rule is prop[@name]="supports'] which will be used to logically link the rule and its condition (and later checks) as supporting evidence for an element in one or more OSCAL model document instances (e.g. a statement of an implemented requirement of a control in a component-definition, but more to follow). NOTE: This supersedes the link/@rel satisfies-partially and satisfies-completely construct recommend in an earlier draft.

[wendellpiez]

Are we allowed to debate names yet? I like the model so far. I like the migration of the settings from link to prop.

Is condition-target a subject? The idea with task/subject, associated-activity/subject is that these are inline descriptions (which could have links) as opposed to simply links to other (first-class) OSCAL objects. As against that use case ('rule' vs description of an assessment task), obviously requiring them to be links to first-class objects has processing implications perhaps good ones. But the boundary seems a bit blurry. (Is a rule an abstraction of a task?)

https://pages.nist.gov/OSCAL/reference/latest/complete/xml-definitions/#/assembly/oscal-assessment-common/task/subject

[aj-stein-nist]

Are we allowed to debate names yet?

Absolutely!

I like the model so far. I like the migration of the settings from link to prop.

Great, thanks for saying so.

Is condition-target a subject? The idea with task/subject, associated-activity/subject is that these are inline descriptions (which could have links) as opposed to simply links to other (first-class) OSCAL objects. As against that use case ('rule' vs description of an assessment task), obviously requiring them to be links to first-class objects has processing implications perhaps good ones. But the boundary seems a bit blurry. (Is a rule an abstraction of a task?)

https://pages.nist.gov/OSCAL/reference/latest/complete/xml-definitions/#/assembly/oscal-assessment-common/task/subject

The reason I moved away from that, @wendellpiez, is it could or could not be an abstraction over an (assessment task), dependent upon other future design decisions about what elements in what models it can refer to; task/subject, associated-activity/subject are constrained in that way. But rules do behave similar to them insomuch as they are " inline descriptions (which could have links) as opposed to simply links to other (first-class) OSCAL objects" but cannot point to a known predetermined list of ref sites. It was hard to reason about this when pointing rules to an associated-activity/subject, for example. This is why I backed out of using the subject and tasks constructs, because they are dependent on referring specific entities with @uuid identifiers that must precisely exist in specific models. Rules might be more haphazard than that.

(Ironically, this whole paragraph is a summary of yesterday's thought process around the id, uuid, id-ref conversation with you and why I struggled to explain why my previous design was hard to practice with an example.)

[aj-stein-nist]

Updated some more examples above. More examples to come, and maybe gathering up documents on use cases more concretely over the next week.

[aj-stein-nist]

@stephenbanghart, last we chatted you had discussed interest in this draft to add recommended features to one or more OSCAL models. The current draft design, with some examples embedded in a component, are available for review and public comment. Feedback welcome!

[aj-stein-nist]

Added some more examples. I am also tracking component-definitions and other XML samples in the following fork branch of the oscal-content repo.

https://github.com/aj-stein-nist/oscal-content/tree/issue-1058-rules

[aj-stein-nist]

In today's developer meeting, Dave discussed an emerging standard around remote attestation of system facts, RATS. I will look into this for potential overlap or improvements that can inform this work, especially around rules and rule checks regarding automated testing to support assessment objectives in some cases.

[aj-stein-nist]

Also in the developer meeting, @JJediny resurfaced (at least for me) InSpec profiles developed, maintained, and operated by MITRE and CMS as part of application of the MITRE Security Automation Framework for instrumenting manual assessment results alongside automated network, cloud infrastructure, and system posture testing.

https://github.com/CMSgov/cms-ars-3.1-moderate-manual-controls-overlay#addressing-manual-controls

Consider looking into some more interesting examples around embedding parameterized rule sets for the manual security testing scenarios.

[aj-stein-nist]

Had a very constructive conversation with @stephenbanghart and will have to work about understanding of overlap/conflicts with the following portions of the assessment-plan and assessment-results models.

• assessment-plan
  • task
    • associated-activity
    • dependency
    • subject
• assessment-results
  • observations

[flickerfly]

I like the idea of bringing this concept to the front. Would a CCI represent a "rule" for those of us in a DISA context?

I'm trying to think about modeling a potential continuous authorization for software. Would a single software pipeline filled with various tools evaluating conditions be represented as its own CDef? In a cATO model, I'm assuming the pipeline gets the authorization and the output of it inherits the provided authorization. If pipeline =/= CDef, how would it be best to represent the relationship of the various tasks within the pipeline to that pipeline? It seems like a pipeline == rule is insufficient in granularity since the pipline represents a host of checks for compliance with a host of rules. Does this fall to dumping a pipeline ID into condition-evaluator\@props for each rule?

Is this in any way related to the idea of mapping controls to things like CCIs that I've heard is coming in 1.1? Thank you for indulging my questions while I do not yet fully grok all that OSCAL is.

[david-waltermire]

@aj-stein-nist and @stephenbanghart Given the outline above and the notes from your conversation, we should consider reusing the actor assembly from the OSCAL assessment plan/results model in place of condition-evaluator. Also, use of assessment-subject might be a useful replacement for condition-target.

[aj-stein-nist]

@david-waltermire-nist, sorry I had not updated comments as I have worked on this. I am trying to work through examples of my own for assessment-plan and assessment-results with two intents:

1. What you requested in the previous comment, considering the move from an existing assembly definitions in the SAP and SAR models.
2. Preparing supporting documents to explain the difference between rule constructs and what else exists in the SAP and SAR models, aka: we have SAP and SAR, why must a rule exist?

[aj-stein-nist]

Some updated direction from community feedback received yesterday:

• Consider parameterization (with set-parameter and parameter) of arguments to use parameters in (for now) what is the condition-evaluator
• Review the supports with ref-id or uuid target approaches when linking from a rule to a target statement in an implemented-requirement of a control-implementation (one example target):
  • One-to-many current approach: link from rule with targets to places like statement in an implemented-requirement of a control-implementation
  • One-to-many alternative approach: One approach is to switch, as given in community feedback, and make the current supported item the source and have that link back to the rule for one or more places, for better decoupling
  • Many-to-many approach: Dave suggested there might be a solution around being more flexible with a mapping construct

[aj-stein-nist]

So I apologize, @flickerfly, I had held off on responding. On some of these questions I still don't have good answers. I held off a while, but I have been thinking about them a lot as of late, so I thought now was the time.

I like the idea of bringing this concept to the front. Would a CCI represent a "rule" for those of us in a DISA context?

I guess yes and no. Funny enough, I had never read the official definition of CCI from the DISA website itself, just used them for a long time. So, what does the website say? It says "[t]he Control Correlation Identifier (CCI) provides a standard identifier and description for each of the singular, actionable statements that comprise an IA control or IA best practice. CCI bridges the gap between high-level policy expressions and low-level technical implementations."

So a rule also wants to be a standard identifier and description (in OSCAL), so we have that in common, yes!

CCIs are "designed to be identify each of the singular, actionable statements that comprise an IA control" and they do seem to do that. "That" here might not be good enough. I am not sure if the wording around statements and controls was intentional, but one of the major challenges with how CCI data (separate of its design) is developed, maintained and operationalized is that it only targets the control. I am surprised to see the definition says "statements of an IA control" but in practice that rarely seems to happen. Often, CCI maps are still high level. Let's look at an example.

With 800-53 Revision 4 AC-2(9), we have some CCIs that are usable: CCI-002140, CCI-002141, et cetera. Notice these go into what people call "sub-controls" and they have to reference content to a parameter in the parent control (AC-2 (e) and its parameter). CCIs don't really get that specific. One of the problems is that lack of specificity, flexibility, and the ability to map into multiple items. I am not sure if the issue was bad data, I usually see just CCI metadata point to AC-2(9) and not AC-2(9).1 and AC-2(9).2. I will keep on examining this, but in a prior working group prior to full-time employment with NIST and this project, I bumped up against this problem a few times.

So, that was a mouthful, what does this boil down from analysis and CCI and real-world usage that I have experienced? Why not just shove CCI in OSCAL?

• CCI is great but not specific enough as it stands for a similar (not identical) intent within OSCAL
• Parameters and how they impact the mapping CCI to NIST RMF 800-53 controls is not explicitly handled (I guess DISA knows they are strongly or maybe absolutely recommending the RMF control parameters and STIG variables are not negotiable?)
• Minor challenge: DISA has still not released CCI mappings upgraded to Rev 5
• Major challenge: not everything will be around STIG and RMF mapping, and the CCI datasets are focused on that so it is not clear outside of operating system hardening baselines it is flexible enough and counterexamples are limited
• Somewhat challenging mechanical issue: in OSCAL 1.0.0-rc2, the <any> construct for using some other XML/JSON/whatever inside OSCAL was removed by community consensus, background details presented and discussed in the model meeting before the decision here

Feedback welcome! But this was a very insightful question you asked, I think I want to put together some information for a later data focusing on the "Why not CCI?" question.

I'm trying to think about modeling a potential continuous authorization for software. Would a single software pipeline filled with various tools evaluating conditions be represented as its own CDef? In a cATO model, I'm assuming the pipeline gets the authorization and the output of it inherits the provided authorization. If pipeline =/= CDef, how would it be best to represent the relationship of the various tasks within the pipeline to that pipeline? It seems like a pipeline == rule is insufficient in granularity since the pipline represents a host of checks for compliance with a host of rules. Does this fall to dumping a pipeline ID into condition-evaluator\@props for each rule?

So this is the question, the more I think about it, I still do not have a good answer. It made me think and go back to the drawing board on some things, hence my lack of response. This came up again yesterday in a review meeting about my current draft design, and there is a question about grouping. I guess for now in this reply my reply is: stay tuned!

Is this in any way related to the idea of mapping controls to things like CCIs that I've heard is coming in 1.1? Thank you for indulging my questions while I do not yet fully grok all that OSCAL is.

Given my comment about a meeting about this yesterday, we are going to determine how that should be handled in this work in the short-term and make it a key requirement of the first design or defer it.