contentctl
contentctl copied to clipboard
splunk
Drilldown Support
This PR has been inspired by the following PR and supercedes it: https://github.com/splunk/contentctl/pull/50
The codebase has changed significantly since that PR was opened, so it was ported to this new PR.
Just thinking aloud, it would be neat if there was an option to add the search as a drilldown by default with a feature flag or a config stanza.
This would help alleviate some annoying behavior associated with adding drilldowns to searches that do not have notable actions so they can be used in Risk Notables.
Just thinking aloud, it would be neat if there was an option to add the search as a drilldown by default with a feature flag or a config stanza.
This would help alleviate some annoying behavior associated with adding drilldowns to searches that do not have notable actions so they can be used in Risk Notables.
Hmmm interesting. What you mean here is to entirely reproduce the search field in the YML as the drilldown.search field, right? It sounds like this is a fairly common use case for you?
Hmmm interesting. What you mean here is to entirely reproduce the
searchfield in the YML as thedrilldown.searchfield, right? It sounds like this is a fairly common use case for you?
Yeah that's right, we do this to all TTP and Anomaly detections as it allows us to quickly jump to the results from the Incident Review page or Risk Notable workbench in order to tweak the filter or our dynamic risk score macros.
Because drilldowns are part of the notable response action, the process of adding the drilldown via the UI can be really annoying. Not just because we have to add the response action and hit save in the case of an Anomaly rule, but also because it appears that we have to wait somewhere between 1-5 mins before we can remove the notable action and hit save again, else the addition of a drilldown doesn't appear to 'stick'.
The abiliity to add these drilldowns by default would be a massive win for us, and will also ensure that any changes to the search will also update the drilldown.
Hmmm interesting. What you mean here is to entirely reproduce the
searchfield in the YML as thedrilldown.searchfield, right? It sounds like this is a fairly common use case for you?Yeah that's right, we do this to all TTP and Anomaly detections as it allows us to quickly jump to the results from the Incident Review page or Risk Notable workbench in order to tweak the filter or our dynamic risk score macros.
Because drilldowns are part of the notable response action, the process of adding the drilldown via the UI can be really annoying. Not just because we have to add the response action and hit save in the case of an Anomaly rule, but also because it appears that we have to wait somewhere between 1-5 mins before we can remove the notable action and hit save again, else the addition of a drilldown doesn't appear to 'stick'.
The abiliity to add these drilldowns by default would be a massive win for us, and will also ensure that any changes to the search will also update the drilldown.
If this is the case, do you see any reason not to just reproduce the search as a drilldown for ANY piece of content (not just Anomaly/TTP) that doesn't declare the drilldown field in its YML? This way, everything gets a generic drilldown (its own search) unless a more specific one is defined (in which case, it gets that one instead).
If this is the case, do you see any reason not to just reproduce the search as a drilldown for ANY piece of content (not just Anomaly/TTP) that doesn't declare the drilldown field in its YML? This way, everything gets a generic drilldown (its own search) unless a more specific one is defined (in which case, it gets that one instead).
I don't see why not myself, as I don't believe that we currently use drilldowns any other way, and as far as I'm aware there's no harm in having them applied to all detection rules.
That said, I could imagine that some users may wish to declare multiple drilldowns, and it would be a shame not to support that, so it might be worth considering multiple drilldown support and a variable replacement-like reference. For example:
drilldown:
- name: %name%
search: %search%
earliest_offset: "$info_min_time$"
latest_offset: "$info_max_time$"
- name: Some other drilldown
search: "some=search"
earliest_offset: "$info_min_time$"
latest_offset: "$info_max_time$"
If this is the case, do you see any reason not to just reproduce the search as a drilldown for ANY piece of content (not just Anomaly/TTP) that doesn't declare the drilldown field in its YML? This way, everything gets a generic drilldown (its own search) unless a more specific one is defined (in which case, it gets that one instead).
I don't see why not myself, as I don't believe that we currently use drilldowns any other way, and as far as I'm aware there's no harm in having them applied to all detection rules.
That said, I could imagine that some users may wish to declare multiple drilldowns, and it would be a shame not to support that, so it might be worth considering multiple drilldown support and a variable replacement-like reference. For example:
drilldown: - name: %name% search: %search% earliest_offset: "$info_min_time$" latest_offset: "$info_max_time$" - name: Some other drilldown search: "some=search" earliest_offset: "$info_min_time$" latest_offset: "$info_max_time$"
Yeah, very cool. Enterprise Security needs to save these in a slightly different serialization format in savedsearches.conf (I just confirmed this experimentally) so I will make some updates to this PR to support an arbitrary number of drilldowns. Probably populating the first one with the search itself with a nice, descriptive name :)
I have closed out this old DRAFT PR in favor of this one since it uses contentctl 4.0 code structure: https://github.com/splunk/contentctl/pull/134 We can refer to that one if we want to bring more of its ideas/constrains into this one.
@pyth0n1c : 100% agreed, i think we should support the multiple drill downs that is compatible with latest ES version.
Here's an example. stanza
action.notable.param.drilldown_searches = [{"name":"drilldown 1","search": ".... | search user = $user$ | search dest = $dest$","earliest_offset":"$info_min_time$","latest_offset":"$info_max_time$"},{"name":"drilldown 2","search":"search 2","earliest_offset":"$info_min_time$","latest_offset":"$info_max_time$"}]
And if I may, while we are making this change, can we change drilldown to drilldown_searchesin the yml, cuz then later if will be easier to add a new key called drilldown_dashboards to support this in the future:
Example stanza for reference:
action.notable.param.drilldown_dashboards = [{"app":"DA-ESS-EndpointProtection","dashboard_id":"endpoint_changes","name":"View process details","tokens":[{"name":"process_name","value":"process_name"}]}, {"app":"DA-ESS-EndpointProtection","dashboard_id":"malware_search","name":"Malware on $dest$","tokens":[{"name":"dest","value":"dest"}]}]
@0xC0FFEEEE : Curious to know, on our end Anomaly detections are only creating risk configs in ss.conf, are you also updating these Anomaly searches in your environment to generate notables?
@patel-bhavin Yes we do add the notable action to Anomaly detections. This can generate a lot of notables, but we tend to tune an Anomaly detection against the previous 30 days worth of data before enabling them.
This approach allowed us to adopt RBA in 12 months of ownership with a very small team.
draft MR with yaml changes : https://github.com/splunk/security_content/pull/3123
NOTE: this adds drilldown as a top-level key!
New security content yaml changes as per our discussion yesterday, have added details to the description of the new PR
https://github.com/splunk/security_content/pull/3130
It is expected that current PRs against security_content repo are failing because the develop branch of security_content does not yet have Drilldowns added