Incorrect `by-id`'s in fedramp moderate profile.

[butler54]

Describe the bug

I ran across this when trying to complete a full profile resolution in the moderate profile. I believe it exists in all profiles as well, however, I have not extensively tested yet.

Context:

ac-12 exists in the moderate profile, ac-12.1 does not.

Using the json profile referring to (part of) the modify statement for ac-12:

{
            "control-id": "ac-12",
            "adds": [
              {
                "position": "starting",
                "by-id": "ac-12",
                "props": [
                  {
                    "name": "CORE",
                    "ns": "https://fedramp.gov/ns/oscal",
                    "value": "true"
                  }
                ]
              },
              {
                "position": "starting",
                "by-id": "ac-12.1_obj",
                "props": [
                  {
                    "name": "response-point",
                    "ns": "https://fedramp.gov/ns/oscal",
                    "value": "You must fill in this response point."
                  },
                  {
                    "name": "method",
                    "value": "EXAMINE",
                    "class": "fedramp"
                  }
                ]
              }
]
}

You will notice a by-id referring to ac-12.1_obj. Which is the top objective level object for ac-12.1. I believe it should be referring to ac-12_obj.1 which is contained within ac-12.

Side note:

I'm not sure whether, if ac-12.1 existed within the moderate profile, this would technically be valid or not (e.g. referring to a by-id in a nested control). My sense is it could be technically valid, but not best practice under those circumstances.

Who is the bug affecting?

Users of OSCAL based profiles.

What is affected by this bug?

Correctness of the profiles.

When does this occur?

When attempting to resolve the profile users will most likely encounter the error

How do we replicate the issue?

Read the files.

Expected behavior (i.e. solution)

{A clear and concise description of what you expected to happen.}

Other Comments

Other known cases in fedramp moderate profile (please excuse the verbose logging).

trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ac-12 and id ac-12.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ac-4 and id ac-4.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ac-4 and id ac-4.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control au-11 and id au-11.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control au-4 and id au-4.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control au-9 and id au-9.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control au-9 and id au-9.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ca-8 and id ca-8.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ca-8 and id ca-8.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-4 and id cm-4.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-5 and id cm-5.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-5 and id cm-5.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-5 and id cm-5.3_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-5 and id cm-5.4_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-5 and id cm-5.5_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cm-5 and id cm-5.6_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cp-10 and id cp-10.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cp-6 and id cp-6.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cp-6 and id cp-6.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cp-8 and id cp-8.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cp-8 and id cp-8.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control cp-8 and id cp-8.3_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ia-2 and id ia-2.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ia-3 and id ia-3.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ia-8 and id ia-8.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ir-3 and id ir-3.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ir-3 and id ir-3.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ir-5 and id ir-5.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ir-7 and id ir-7.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ir-7 and id ir-7.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ma-3 and id ma-3.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ma-3 and id ma-3.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ma-3 and id ma-3.3_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ma-6 and id ma-6.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ma-6 and id ma-6.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control ma-6 and id ma-6.3_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control mp-7 and id mp-7.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control mp-7 and id mp-7.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control pe-12 and id pe-12.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control pe-13 and id pe-13.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control pe-13 and id pe-13.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sa-4 and id sa-4.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-12 and id sc-12.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-12 and id sc-12.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-28 and id sc-28.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-28 and id sc-28.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-5 and id sc-5.1_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-5 and id sc-5.2_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control sc-5 and id sc-5.3_obj
trestle.core.profile_resolver:501 ERROR: Did not find the correct ID to add props for control si-7 and id si-7.2_obj

[ohsh6o]

I can examine the additional output of this but it seems the first, AC-12.1 is probably a legitimate reference in the high resolved profile catalog, but not the moderate. I am not sure how that slipped through, but I will need to examine further. This screenshots that document in the FedRAMP High Baseline SSP Template.

Can you please link to how you used compliance-trestle or separate code to generate this error with the given moderate resolved profile catalog you provided?

cc @volpet2014 for awareness.

[butler54]

@ohsh6o I think my confusion still stands for HIGH as:

1. The convention within the profile (excluding the exceptions listed) is that objectives in sub-controls which are the subject of a modify statement are referred to with the sub-control as the control ID. e.g.
2. It seems like the standard approach is when we have response points they are at a consistent level. When examining fedramp high (and ac-12 / ac-12.1 as examples:
   1. ac-12 adds (objective) response points one of two of second level objectives for ac-12 (ac-12_obj.2 and not ac-12_obj.1).
   2. ac-12 then adds response points to the top level objective of ac-12.1 (ac-12.1_obj)
   3. When we examine ac-12.1 all of the meaningful line items (ac-12.1.a_obj.1, ac-12.1.a_obj.2, ac-12.1.b_obj)
3. I don't believe Determine If: (the prose for for ac-12.1_obj is meant to actually have a response`.

I'll put up a script in the next few days once the changes are merged (at least) into our develop branch.

[volpet2014]

This issue has been resolved by PR #252 which is pending merge. Once this merge occurs, this issue should be resolved as it was related to mis-aligned ids in the profile and resolved-profile-catalog