heimdall2 Hdf2ckl severity

Fix to #5842.

May 31 '24 14:05 kemley76

The hdf2ckl export works properly, however, the Details tab should be:

Severity: (tag.severity value)

Impact: (Impact value)

Jun 03 '24 17:06 ejaronne

I have made more changes regarding this issue. This includes:

adding a severity tag upon importing a Checklist file
using a control's severity tag to compute its severity rather than defaulting to use only the impact value
compute the amount of "impact bubbles" in the results view with the impact rather than the severity
be able to sort the controls by impact when clicking the impact column header
be able to show falsy values in the details (like when there is impact of 0)

Done in previous commit, but not listed out

use severity tag (if available) when exporting form HDF to CKL

This shows the sorting and bubble usage. The bubbles indicate impact, the text indicates severity. Screenshot 2024-06-05 at 11 48 52 AM

Jun 05 '24 16:06 kemley76

The original issue was caused by two things:

When a control has a severity tag, Heimdall didn't care and computed the severity it displays with the impact, which can differ from this tag
When exporting from hdf to checklist, the severity in the resulting checklist was computed with the hdf's impact and disregarded any severity tags. These have been addressed

Additional fixes that were addressed that may be beyond the scope of the original issue, but still needed to be addressed:

Compute a control's severity based on

Any severity override (checklist vuln tag or hdf control tag)
Any severity tag (checklist tag or hdf control tag)
Impact

Display severity override information in results table Screenshot 2024-06-14 at 4 31 46 PM Display severity override information in details view Screenshot 2024-06-14 at 4 32 37 PM

It was also noticed that these changes affected the center "Severity Counts" ring graph. Before the "severities" in the graph were computed directly from the impact. Now, if there is a severity tag or severity override, that will be shown in the graph instead. When importing checklists, it is now impossible to see severity: none in this graph since severity: none does not exist in the checklist world.

Jun 14 '24 20:06 kemley76

One last behavioral change was made here: Upon exporting form hdf to checklist, severity tags none and critical now map to low and high respectively. This has the side effect that exporting from hdf to ckl and then back to hdf could result in different severity levels if they were originally none or critical.

Jun 17 '24 14:06 kemley76

The impact and severity columns have now been separated and are being displayed with color coded chips. Results View Changes

Jun 17 '24 18:06 kemley76

@kemley76 will you be coding a "filter to results with severity overrides", similar to the "filter to profile errors"?

Jun 18 '24 17:06 ejaronne

@ejaronne I previously looked into this and I think it might be too much for this issue. Currently, the system with the "filter to (profile) errors" or "filter to waived (controls)" is set up under the exact same component system as with the four main colored filter bars (passed, failed, N/A, not reviewed). Since "overridden" is not really a control status, this would make it difficult to implement under that system. Another option might be to add this filtering capabilities to the search bar at the top or to the switches on the "results view data" table, but still, might not be the most straightforward thing to implement.

Jun 18 '24 19:06 kemley76

@ejaronne I previously looked into this and I think it might be too much for this issue. Currently, the system with the "filter to (profile) errors" or "filter to waived (controls)" is set up under the exact same component system as with the four main colored filter bars (passed, failed, N/A, not reviewed). Since "overridden" is not really a control status, this would make it difficult to implement under that system. Another option might be to add this filtering capabilities to the search bar at the top or to the switches on the "results view data" table, but still, might not be the most straightforward thing to implement.

Sorry I meant cosmetically the same, but logic based on severity override situation.

Jun 19 '24 17:06 ejaronne

I have created a way to filter to controls with severity overrides by filtering for the specific severity override tag. Screenshot 2024-06-24 at 3 48 03 PM

This involved adding support to filter controls by the presence of specific tags, which is now documented in the search bar help menu.

Jun 24 '24 19:06 kemley76

Yet again, I am a little confused on what the best way to do this is. Currently, I have reverted back to using two columns and I'm not too sure how to go about things.

Points of confusion: 1: Why do we even have to show severity to begin with? I understand that many security formats use it to represent how bad a defect is if it was exploited, but OHDF represents this mostly with impact and severity is not mentioned in the schema. Despite this, severity was created as just a word version of impact (inspecJS says that here) until I started messing with it by allowing them to differ if a severity tag was present.

2: How should we display impact/severity? Before I started this issue, there was a column that displayed impact . This is what Heimdall users are familiar with. If we introduce a new, slightly different severity column, that would be confusing, right? Do we need to deal with the severity tags (the only way that severity differs from impact)? Do people care about the severity tags? In the case where the severity tag and impact differ, I can indicate this by adding a little delta symbol + tooltip to show up in the impact column (checklist's severity override can trigger this too). I had this system setup before. This makes all information available in a single column.

3: How do we compute severity and impact? I think if we have severity computed 1st by severity override tag, 2nd by severity tag, and 3rd by impact, it would allow mappers, particularly checklist, to add severity override tags that can be indicated in the UI.

Jun 24 '24 21:06 kemley76

Everything that was changed

Updated a bunch of tests

Filtering severity override test
Ckl mapper and reverse mapper tests with severity overrides
Cypress test to ensure that severity overrides are rendered in results table

Changed mapper behavior

when loading an HDF file, inspecJS now uses severity tag to compute severity if present
when loading a ckl file
- severity and severity override are used to compute impact
- severity override tags are added if severity override is given
- severity override justification tag is added if in checklist
- severity tag is always added now
when converting hdf to ckl
- severity tag is used to compute severity (falls back to use impact if no tag present)
- severity override tag is used to compute severity override in checklist
- if it wouldn’t be able to go back to hdf properly, the data is stored in checklist's third-party tools section

Updated Results Table

Changed impact column to severity column (now on the right of title)
- shows essentially the same information (bubbles + word -> colored word in chip)
- information is only different for user if input file has a severity tag that differs from impact (either from mapping that adds one or from inspec profile with explicit tag)
- if there is a severity override (currently only comes from checklist files, it shows a delta symbol with a tooltip that shows what it changed from and the justification)
details page
- if there is a severity tag, it is shown (falls back to internal severity which is typically computed from impact)
  - this makes it clear when there is a severity tag and severity override tag and what the difference is
- fixed bug where impact 0 would not show up in details view

Filtering

added ability to filter by the presence of a tag (tags:”severityoverride,nist”)
added popup (like profile errors warning) that appears when a loaded profile has requirements with severity overrides (there is button to filter to them)

Jun 28 '24 17:06 kemley76