Add CSVs and automate detection artifact generation

[lawndoc]

Add CSVs to support additional detection use cases

With KQL (M365 Defender and Sentinel), you can import CSVs from a web address as a table to use in hunting/detection queries. Adding CSVs to this project would allow defenders to correlate this repo's information with the DeviceFileCertificateInfo table from MDE to pivot off of for additional hunting. Here is an example query:

let Certs = materialize(
    (externaldata(response: string) [@"https://github.com/lawndoc/lolcerts/raw/main/csv/leaked.csv"] with (format="txt"))
    | where response !startswith "name,status,source"
    | extend data = parse_csv(response)
    | extend
        Name = tostring(data[0]),
        Status = tostring(data[1]),
        Source = tostring(data[2]),
        Description = tostring(data[3]),
        References = tostring(data[4]),
        Date = tostring(data[5]),
        Author = tostring(data[6]),
        Issuer = tostring(data[7]),
        Timestamp = tostring(data[8]),
        Serial = replace_string(tostring(data[9]), ":", "")
    | project-away response, data
);
Certs
| join kind=innerunique DeviceFileCertificateInfo on $left.Serial == $right.CertificateSerialNumber

 

Automate CSV and YARA generation

The workflow added in .github/workflows/generate_artifacts.yml will automatically update the YARA and CSV files when there are changes to the leaked or malicious directories pushed to the main branch.

This requires your GitHub Token to have Read and Write permissions in your GitHub Actions settings so it can commit the files back to the repo.

[RiccardoAncarani]

This is amazing! Thanks so much for this, I'll review the changes this week but looks great so far.

[lawndoc]

I made one small cosmetic change, it seems the GitHub bot email I used to commit back to the repo in the GitHub Actions workflow was for the GitHub Web Flow account (which isn't quite accurate).

[hjl-csis]

Hi, I'm not associated with this repository, but would also love to see a CSV export as well, so thank you very much for this PR!

I'd suggest to make use of Pythons csv. writer - that way you don't have to worry about escaping or removing special characters, as the Python standard library will do the escaping and quoting for you, if necessary. :)

[lawndoc]

Good thought @hjl-csis, I'll update my fork to use the csv module.

[lawndoc]

@RiccardoAncarani - I finally got around to modifying the script to use csv.writer as suggested by @hjl-csis. This PR should be ready for your review.

[lawndoc]

@RiccardoAncarani - just checking in, when do you think you'll be able to take a look at this? If you guys don't have the time to maintain this project, I'd be happy to fork it or assist with the daily stuff.

[RiccardoAncarani]

Hi,

Apologies for the time it took us to review this. The code looks good and seems to be working ok. Will add you to the contributors as well.