automate icon indicating copy to clipboard operation
automate copied to clipboard

Reporting exports differ between CSV and JSON output when deep searching on a specific control in a specific profile

Open chef-davin opened this issue 3 years ago • 0 comments

Describe the bug

When using deep searching to try to view the results of nodes that have failed a specific control within a specific profile, exporting the report gives you different results whether you export a JSON or a CSV. This is true in the UI as well as the API.

Using the UI as an example. If I filter on a specific compliance profile, then filter on a specific control within that profile, then look at the nodes by status (failed, passed, etc...) in the UI I see a subset of the nodes returned based on whether they match the status for that specific control. If I select a control where all nodes have passed the control, and then look at the node list but filter on "failed nodes", the list of nodes is empty because none of the nodes have failed the control. This is expected and right.

However, when I export a report based on my filtering, I get different results exporting JSON and exporting CSV. The JSON export will return the control results for any nodes that have failed ANY control in the profile. Whereas the CSV export will return an empty set with just the headers because it's only returning the nodes that have failed the specific control that I'm filtering on. Ideally, both the JSON and CSV results should be returning empty sets in this instance.

To Reproduce

Steps to reproduce the behavior:

  1. Stand up an A2 instance
  2. use chef-load to load compliance data using the compliance sample data provided in the chef-load repo. (in my reproduction I loaded 100 nodes)
  3. In the UI, go to the compliance tab. Add a filter on the CIS Ubuntu 14.04 LTS Server Benchmark Level 1 profile
  4. Next add a filter on the xccdf_org.cisecurity.benchmarks_rule_1.1_Install_Updates_Patches_and_Additional_Security_Software control.
  5. Click on nodes - There should be an amount equal to whatever number you passed to chef-load
  6. Select failed nodes (you should see zero nodes in the list now)
  7. export a JSON report
  8. export a CSV report
  9. compare the differences

Alternatively, you could do this through the API. Use this to get a JSON report with the same profile ID and control:

curl --request POST \
  --url https://chef-automate.test/api/v0/compliance/reporting/export \
  --header 'Content-Type: application/json' \
  --header 'api-token: <INSERT_API_TOKEN>' \
  --data '{
	"filters": [
		{
			"type": "profile_id",
			"values": [
				"9880de490ba6b9113431663f50bec6b1a47d7d1e64c7590fd54c42f1c9b42c2e"
			]
		},
		{
			"type": "control",
			"values": [
				"xccdf_org.cisecurity.benchmarks_rule_1.1_Install_Updates_Patches_and_Additional_Security_Software"
			]
		},
        {
			"type": "status",
			"values": [
				"failed"
			]
		}
	],
	"type": "json"
}'

And use this to get a CSV report:

curl --request POST \
  --url https://chef-automate.test/api/v0/compliance/reporting/export \
  --header 'Content-Type: application/json' \
  --header 'api-token: <INSERT_API_TOKEN>' \
  --data '{
	"filters": [
		{
			"type": "profile_id",
			"values": [
				"9880de490ba6b9113431663f50bec6b1a47d7d1e64c7590fd54c42f1c9b42c2e"
			]
		},
		{
			"type": "control",
			"values": [
				"xccdf_org.cisecurity.benchmarks_rule_1.1_Install_Updates_Patches_and_Additional_Security_Software"
			]
		},
        {
			"type": "status",
			"values": [
				"failed"
			]
		}
	],
	"type": "csv"
}'

Expected behavior

Both the JSON report and the CSV report should have 0 results as they should both be returning results based on the control status, and not the profile status.

Versions (please complete the following information):

  • OS: Mac OS X 10.14 (client), CentOS 7 (a2 server)
  • Browser: chrome
  • Automate Build Number: 20210726160836 (dev channel), 20210727104144 (current channel)

Additional context

Add any other context about the problem here.

Aha! Link: https://chef.aha.io/epics/SH-E-743

chef-davin avatar Aug 02 '21 15:08 chef-davin