Azure-Sentinel icon indicating copy to clipboard operation
Azure-Sentinel copied to clipboard

Microsoft Sentinel - Delay for Automation Rule execution

Open rtdamiani opened this issue 1 year ago • 5 comments

For a few days I am analyzing a scenario in Sentinel incidents.

In the environment I have some Automation Rules for executing a playbook, closing an incident, adding a tag.

When an incident originates from Defender (Endpoint, Entra ID Protection, Office 365) the Automation Rule takes 5 or minutes to execute.

When an incident originates from Analytic Rules, the execution of the Automation Rule is practically instantaneous.

rtdamiani avatar Feb 02 '24 19:02 rtdamiani

Hi @rtdamiani, Thanks for flagging this issue, we will investigate this issue and get back to you with some updates by 08-02-2024. Meanwhile, could you please share some more details about this issue, like which automation rules or solution you are facing issue. Thanks! Thanks!

v-sudkharat avatar Feb 05 '24 06:02 v-sudkharat

Hi @v-sudkharat , I have attached two files.

print1: incident created by an analytic rule print2: incident created by Defender

When an incident originates from Defender, the Automation Rule presents a delay to execute the playbook, add a tag or close the incident.

print1 print2 print3

rtdamiani avatar Feb 05 '24 11:02 rtdamiani

@rtdamiani, thanks for providing details with us. we will check on this issue and get back to you with some updates. Thanks!

v-sudkharat avatar Feb 06 '24 08:02 v-sudkharat

Hi @v-sudkharat. Is there any feedback on this issue?

rtdamiani avatar Feb 09 '24 00:02 rtdamiani

Hi @rtdamiani, We are still working on investigating this issue by checking the respective playbooks. Meantime, could you please share you mail id with us? so if required we can connect with you via a call. Additionally, Could you please check while configuring the playbook for which trigger has been set - image

And also i can see in you 2nd print screenshot, the automation rule adding the comment before the tag. so can you please let us know is that part of configuration - image

Thanks!

v-sudkharat avatar Feb 09 '24 07:02 v-sudkharat

Hi @rtdamiani. Hope you are doing well. We are waiting for your response on above comment. Thanks!

v-sudkharat avatar Feb 13 '24 06:02 v-sudkharat

Hi, @v-sudkharat thanks for sharing your analysis.

Automarion rule is configured with the trigger "When incident is created".

automation rule

The settings for LogicApp "Enrich-Incident-Alienvault-OTX" haven't been changed. Comment and tag are added by logicapp.

logicapp

My email is [email protected]

rtdamiani avatar Feb 15 '24 00:02 rtdamiani

@rtdamiani, thank you for your response. We will check in this and get back to you by - 20-02-2024. Thanks!

v-sudkharat avatar Feb 15 '24 12:02 v-sudkharat

Hi @rtdamiani, We have sent a mail to you. Could you please check it and share your response. Thanks!

v-sudkharat avatar Feb 20 '24 11:02 v-sudkharat

Hi @rtdamiani, we are waiting for your response on above comment. Thanks!

v-sudkharat avatar Feb 22 '24 10:02 v-sudkharat

Hi @v-sudkharat, email answered now. thanks.

rtdamiani avatar Feb 22 '24 11:02 rtdamiani

Hi @rtdamiani, Could you please let us know if this issue gets resolved. So we can close this issue from GitHub. Thanks!

v-sudkharat avatar Feb 29 '24 07:02 v-sudkharat

Hi @v-sudkharat

Can you add here a print of an automation rule being executed in an incident originating from Defender in your environment?

rtdamiani avatar Feb 29 '24 15:02 rtdamiani

Hi @rtdamiani, currently our environment does not have that playbook configured, so due to that we can't add print over github. Thanks!

v-sudkharat avatar Mar 01 '24 06:03 v-sudkharat

Hi @v-sudkharat

I recommend you read again the email I sent to you on 02-22 at 08:53 AM.

In the email I explain and provide evidence that the delay to execute automations isn't related to the playbook execution, but to any Automation Rule executed in Defender incidents. The evidence contains a print of an incident originating from Entra ID Identity Protection. An Automation Rule has been configured in the environment to just add a comment and close the incident.

The print shows the Activity Log of the incident. When the incident is created by Defender, the Automation Rule takes an average of 7 minutes to start. In another print with Activity Log of an incident generated by Analytic Rule, the Automation Rule is executed in the same minute.

The delay I'm talking about isn't related to any automation rule (with or without a playbook) in the 3 tenants I manage.

Again I ask: Is this behavior normal or is it a possible problem in Microsoft Sentinel?

Below are the prints:

Incident created by analytic rule

teste1

teste2

Incident created by Defender

teste4

teste3

So, could you please create an Automation Rule and configure it to run on Defender incidents and post the Activity Log printout here?

rtdamiani avatar Mar 01 '24 11:03 rtdamiani

Hi @rtdamiani, we will check on it and get back to you with some updates via a GitHub comment or Mail. Thanks!

v-sudkharat avatar Mar 04 '24 12:03 v-sudkharat

Hi @rtdamiani, After our last call, we have replicated this issue in our environment, please find below screenshot for reference in which there is delay up to 10 min after creating an incident by Microsoft Defender XDR.

image

Sharing the MS doc screenshot and link for reference - Link - https://learn.microsoft.com/en-us/azure/sentinel/microsoft-365-defender-sentinel-integration image

Please let us know if you still have any queries further.

v-sudkharat avatar Mar 08 '24 12:03 v-sudkharat

@v-sudkharat The issue isn't about the incident taking 10 minutes to show up within Sentinel, but that it takes >5 minutes to start executing an automation rule. It should be instantaneous as for other sources. We are seeing the same behavior. If this is working as designed please confirm that.

Over that 5 minute period an incident can be picked up by an agent wasting time because it would be handled by an automation rule otherwise.

Kaloszer avatar Mar 11 '24 08:03 Kaloszer

Hey @rtdamiani /@Kaloszer, We have shared this feedback with our respective concern team, Once we receive the update from them, will update you. Thanks!

v-sudkharat avatar Mar 12 '24 11:03 v-sudkharat

Hi @rtdamiani / @Kaloszer. Thanks for the patience so far. Even we had this issue puzzled, so we reached out to Automation team for the clarification.
This confusion is caused because the timestamp for the first activity log "Incident was created" is the timestamp when this incident is created in Microsoft Defender. Let me explain from above screenshot that you have shared.

  1. An incident is created in Microsoft Defender at 5.02 PM
  2. Through Microsoft Defender connector it took around 6 minutes to appear in Microsoft Sentinel at 5.08 PM
  3. This generated activity log "Incident was created" but timestamp used 5.02 PM when it was created in Microsoft Defender
  4. At the same time automation rule triggered.

So there is no delay in automation trigger. The choice to use the Microsoft Defender incident timestamp is intentional, to keep data at both the places Defender and Sentinel coherent.

rahul0216 avatar Mar 13 '24 16:03 rahul0216

Hi, @rahul0216 thanks for your feedback.

I understood what you explained about the time the incident is created in Defender being different from the time the incident is created in Sentinel.

But the time that appears in "Incident was created" in "Incident activity log" is the "CreatedTime" of the incident in Sentinel. The time the incident was created in Defender is the "FirstActivityTime".

Using the SecurityIncident table to collect the data:

Title: Email messages containing malicious file removed after delivery​ IncidentNumber: 19019 FirstActivityTime: 3/12/2024, 10:13:56.891 AM (Incident time was created in Defender) CreatedTime: 3/12/2024, 10:31:49.940 AM (Incident time was created in Sentinel) TimeToSyncSentinel: 18 minutes

Is the information about the times of incidents created correct?

tableincident

Now showing the incident and Activity log:

Incident was created: 03/12/24, 10:31 AM == CreatedTime: 3/12/2024, 10:31:49.940 AM

activity

Automation rule executed only to close the incident, without playbook execution.

automation rule

The Delay is related to the execution of any automation in Defender incidents.

The time to synchronize the incident between Defender and Sentinel was 18 minutes and the delay to start the automation was 7 minutes.

rtdamiani avatar Mar 14 '24 01:03 rtdamiani

No exactly, image

rahul0216 avatar Mar 14 '24 05:03 rahul0216

So as I understand it the incident was created at 10:31 in Defender > Synced to Sentinel at that 10:38 mark and then the Automation rule was invoked?

Ergo there's no timestamp that would actually reflect when the incident showed up in Sentinel as the former ( Defender timestamp ) is being used as a point of reference in time?

That would mean that this is a non issue as the automation rule actually does invoke automatically as soon as it's available in sentinel :)?

If this is the case this should be referenced somewhere in documentation as this is going to be coming up whenever someone 'notices' this, sometime in the future.

Kaloszer avatar Mar 14 '24 07:03 Kaloszer

Title: Email messages containing malicious file removed after delivery IncidentNumber: 19019 ProviderIncidentId: 16296 FirstActivityTime: 3/12/2024, 10:13:56.891 AM (Incident time was created in Defender) CreatedTime: 3/12/2024, 10:31:49.940 AM (Incident time was created in Sentinel)

inc sentinel v2

inc defender

activity v2

rtdamiani avatar Mar 14 '24 13:03 rtdamiani

Hi @rtdamiani / @Kaloszer, We understand your question, The Creation time which you see after the Incident get generated it is Incident creation time in Defender, not the incident created into the sentinel. As mentioned into the MS doc, the sync having upto 10 min of delay. So once the Incident is fully created into the Sentinel, the automation rule will run without having any delay. as @rahul0216, added into above comment, to keep this data sync in both the places, the timestamp has been chosen for it.

Hope we have clear this. Let us know if your issue gets resolved so we can close this issue from GitHub. Thanks!

v-sudkharat avatar Mar 15 '24 13:03 v-sudkharat

@rtdamiani / @Kaloszer, We are closing this issue. If you still need support for this issue, feel free to re-open it any time. Thank you for your co-operation.

v-sudkharat avatar Mar 18 '24 09:03 v-sudkharat