Azure-Sentinel icon indicating copy to clipboard operation
Azure-Sentinel copied to clipboard

Azurekid - ConvertFrom-Asim

Open azurekid opened this issue 1 year ago • 25 comments

Required items, please complete

Change(s):

  • ConvertFrom-ASim.ps1

Reason for Change(s):

  • Added a script for users to easily create an ARM template from an ASIM parser using PowerShell

Version Updated:

  • N/A

Testing Completed:

  • Added example in examples folder

Checked that the validations are passing and have addressed any issues that are present:

  • Yes

azurekid avatar Aug 25 '23 09:08 azurekid

friendly reminder to pick this us.

azurekid avatar Sep 05 '23 08:09 azurekid

@azurekid Can you please add description for why we need this? For every schema we already have deploy to azure feature https://github.com/Azure/Azure-Sentinel/tree/master/Parsers/ASimDns

vakohl avatar Oct 11 '23 13:10 vakohl

Hi Varun,

The reason I have created this, is because I regularly got questions on how to deploy the MS Sentinel github files.

On one of my projects (Achmea) we are building our own ASIM parsers and functions, and trying to stay as close as possible to the Microsoft standards

Therefore als parsers and functions are created in the YAML format. This function can be used to create deployable artifacts from the yaml files so these can be deployed through a DevOps or GitHub pipeline.

This is also the reason for my order PR where I updated all the ASIm templates

azurekid avatar Oct 11 '23 17:10 azurekid

Hey Guys, Anything I can do to get this PR going?

azurekid avatar Oct 25 '23 12:10 azurekid

@azurekid Customers can deploy the existing ASIM parser for each schema, e.g. through the link I shared. image

We make every parser ARM deployable as soon we get them merged on github. With this, every parser you see under their schemas are deployable already using the link I shared. I'm trying to understand when someone would need this script? Is this going to be used when someone like to deploy a custom parser they built?

vakohl avatar Oct 25 '23 13:10 vakohl

@azurekid Customers can deploy the existing ASIM parser for each schema, e.g. through the link I shared. image

We make every parser ARM deployable as soon we get them merged on github. With this, every parser you see under their schemas are deployable already using the link I shared. I'm trying to understand when someone would need this script? Is this going to be used when someone like to deploy a custom parser they built?

Hi,

This is indeed the case. For the projects I am working on, we are creating a lot of custom ASIM parsers like:

  • BeyondTrust
  • Blue Cedar Networks
  • CommVault
  • Infoblox NIOS
  • Layer 7 SecureSpan Gateway
  • NetApp
  • Radware
  • F5
  • Darktrace
  • Microsoft 365 Defender
  • etc.

We have build over 60 ASIM parsers and converted these to deployable ARM templates using this script.

The whole idea behind this script is that organizations that build parsers or custom functions can easily deploy them and develop occurring the Microsoft Standards in YAML format, so these can easily be shared with the community.

azurekid avatar Oct 25 '23 15:10 azurekid

@azurekid Sounds good. I'll take a look.

vakohl avatar Oct 26 '23 03:10 vakohl

Hi @vakohl Is there anything you are still waiting for from my side? Please let me know 🙏🏼

azurekid avatar Nov 30 '23 05:11 azurekid

Hi @azurekid, We're waiting for an update on this. Thanks

v-atulyadav avatar Dec 04 '23 11:12 v-atulyadav

Hi @azurekid, We're waiting for an update on this. Thanks

Now in the testing phase

azurekid avatar Dec 04 '23 12:12 azurekid

Noted @azurekid.

v-atulyadav avatar Dec 04 '23 13:12 v-atulyadav

Noted @azurekid.

parameters added and tested. updated the readme reflecting the parameter aliasses.

azurekid avatar Dec 04 '23 18:12 azurekid

Thanks @azurekid.

v-atulyadav avatar Dec 05 '23 12:12 v-atulyadav

@azurekid sorry for late response on this PR. I tried to deploy template generated by this script. Seems there's something wrong with the script. The output json templates fails during custom deployment. Can you please re-test with all supported parameters?

vakohl avatar Mar 08 '24 11:03 vakohl

No worries,

Can you please share an error message or the generated template so I can see where it goes wrong. Will also check it today and come back to you

azurekid avatar Mar 09 '24 09:03 azurekid

No worries,

Can you please share an error message or the generated template so I can see where it goes wrong. Will also check it today and come back to you

@azurekid seems like the generated json template has some text that need to be fixed. Just try generating the template using following command and try deploying the json template as custom deployment on Azure. ConvertFrom-ASim.ps1 -FilesPath 'C:\Users\RogierDijkman\Parsers\vimAuditEventAzureAdminActivity.yaml'

image

vakohl avatar Mar 26 '24 14:03 vakohl

Hi @azurekid, Please check response from @vakohl on your asks. Thanks

v-atulyadav avatar Apr 03 '24 04:04 v-atulyadav

Hi @azurekid, We are waiting for your response. Thanks

v-atulyadav avatar Apr 05 '24 04:04 v-atulyadav

Hi @azurekid, We are waiting for your response. Thanks

Hi @v-atulyadav, After extensive testing I was able to identify the issue. for ASIM files very thing worked as expected, but when using a vim an error occurred. This was due to parameter parsing for string values. The KQL validator expects string='*' but the arm template contained string=* and was therefor invalid.

This has been fixed in the file on line 119 functionParameters = $stringParams.replace("string=*", "string='*'").trim(', ')

azurekid avatar Apr 08 '24 11:04 azurekid

Thanks @azurekid. Could you please update your branch by taking master's latest.

v-atulyadav avatar Apr 12 '24 04:04 v-atulyadav

Hi @azurekid, Validations are stuck in queue; please take the latest from the master's branch and push it again. Thanks

v-atulyadav avatar Apr 16 '24 09:04 v-atulyadav

Hi @azurekid, Validations are stuck in queue; please take the latest from the master's branch and push it again. Thanks

All tests are passed 👍

azurekid avatar Apr 16 '24 15:04 azurekid

@azurekid I still couldn't deploy. Used this command: .\ConvertFrom-ASim.ps1 -FilesPath "C:\Users\vimAuditEventMicrosoftExchangeAdmin365.yaml"

image

vakohl avatar Apr 16 '24 17:04 vakohl

Hi @azurekid, Please check above comments. Thanks

v-atulyadav avatar Apr 18 '24 09:04 v-atulyadav

Hi @azurekid, Please check above comments. Thanks

Hi, I have been reverse engineering what is going wrong as it did work properly before without any real changes in the code. Seems like there has been some updates on the API and now doesn't recognize semantic versions as integers.

Updated the code for versioning and synced with the latest master.

Thanks for you patience ;-)

image

azurekid avatar Apr 22 '24 07:04 azurekid