Add entity groups

[ryan-carroll-graylog]

Description

Adds base functionality for entity grouping/tagging/categorizing.

Closes:

• https://github.com/Graylog2/graylog-plugin-enterprise/issues/7415
• https://github.com/Graylog2/graylog-plugin-enterprise/issues/7416

Motivation and Context

See https://github.com/Graylog2/graylog-plugin-enterprise/issues/7385 for details and pitch.

How Has This Been Tested?

Locally using API browser.

Example request to create a new entity group:

{
  "entities": {"assets" : ["789", "777"], "sigma_rules" : ["456", "888"]},
  "name": "category 1"
}

For the first pass only two entity type values are recognized (chosen simply to be able to test this effectively): assets and sigma_rules.

Screenshots (if appropriate):

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [ ] Refactoring (non-breaking change)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• [X] My code follows the code style of this project.
• [ ] My change requires a change to the documentation.
• [ ] I have updated the documentation accordingly.
• [X] I have read the CONTRIBUTING document.
• [ ] I have added tests to cover my changes.

[kingzacko1]

@bernd this is the backend groundwork of the Entity Groups (name we've landed on for the Content Tagging/Categorization feature). Idea is for the DTO to look something like (with empty arrays omitted):

[
	{
		"_id": "661d3e54742f917acb3c675b",
		"_scope": "ILLUMINATE",
		"name": "Windows Compliance",
		"entities": {
			"event_definitions": [],
			"sigma_rules": ["661d3e52742f917acb3c675a", "661d3e52742f917acb3c675b", "661d3e52742f917acb3c675c"],
			"anomaly_detectors": [],
			"dashboards": [ "661d3e54742f917acb3c675a", "661d3e54742f917acb3c674c", "661d3e54742f917acb3c6888" ]
			"saved_searches": [],
			"streams": [ "651d3e54742f917acb3c675a" ],
			"content_packs": []
		}
	},
	{
		"_id": "661d3e54742f917acb3c671f",
		"_scope": "DEFAULT",
		"name": "User's Custom Group",
		"entities": {
			"event_definitions": [],
			"sigma_rules": [],
			"anomaly_detectors": [],
			"dashboards": [ "661d3eaaaa2f917acb3c675a", "661d3eaaaa2f917acb3c674c", "661d3eaaaa2f917acb3c6888" ]
			"saved_searches": [],
			"streams": [],
			"content_packs": ["661d3eaaaa2f917acb3c6fec", "661d3eaaaa2f917acbacd75b"]
		}
	}
]

[ryan-carroll-graylog]

Really appreciate the review and feedback @kroepke! I believe I've addressed everything you noted but definitely let me know if I missed anything.

[ryan-carroll-graylog]

Thanks for all the comments @kroepke! A lot of great info I believe I've addressed here, and will apply across the board elsewhere too.

[ryan-carroll-graylog]

Thanks for reviewing with the latest @danotorrey! Somehow I missed the categories audit logs last time around. Pushed some updates here and opened an enterprise PR for the audit log formatting: https://github.com/Graylog2/graylog-plugin-enterprise/pull/7585

[ryan-carroll-graylog]

@kroepke I believed I've addressed all the PR feedback here. Do you see any issues merging this once we get the last SecDev reviews so we can push forward with front end development?

We've got this behind a feature flag so we can also safely make changes in the future as needed without any major disruptions.

[danotorrey]

We discussed this with the Architecture team, and identified that we are putting the general Entity Groups solution on hold. We may pursue it further later, as a more general way to link entities of different types into a group with potential additional properties such as permissions etc.

In the short-term, we will pursue https://github.com/Graylog2/graylog-plugin-enterprise/issues/7926 instead.

[bernd]

We discussed this with the Architecture team, and identified that we are putting the general Entity Groups solution on hold. We may pursue it further later, as a more general way to link entities of different types into a group with potential additional properties such as permissions etc.

In the short-term, we will pursue Graylog2/graylog-plugin-enterprise#7926 instead.

I am changing the PR to draft status.

[ryan-carroll-graylog]

Closing as we went with a different solution for this.