proposal on the rules

[ruppde]

hi,

did a test drive with your yara rules and while they find malware and nasty things, they just produce too many false positives, to be usable. The ReactOS live CD has 144 hits, the /usr/sbin of debian has 125 hits. So scanning a complete hard drive of a system infected with maybe 3 malware files would produce something like 10.000 false positives. there's just no way to find those needles in the haystack.

proposals to bring that number down:

• scan some goodware and remove the strings, which just hit too often
• • decrease the translation of * from {0,1000} to {0,100} in https://github.com/mthcht/ThreatHunting-Keywords-yara-rules/blob/1523732cfc3c555323e4b47d890e107910c47b0e/_utils/create_yara_rules.py#L110 because a 1000 chars gap is usually too much
• for rules with many strings, it might be ok to switch the yara condition from any of them to 2 of them
• for rules with few strings, consider merging multiple of them with a similar topic into one rule and also go for 2 of them. for example there is string18_cat_greyware_tool_keyword: cat /etc/passwd, which will also be in lots of legitimate scripts. but if there's also string31_net_greyware_tool_keyword: net localgroup admin in the same file, that's rather unusual.
• add a condition of filesize < 10MB to avoid matching on huge legitimate files, which just contain many many strings
• for windows binary hacktools, add a condition of "uint16(0) == 0x5a4d" to match only binaries. otherwise the rules will also match on e.g. emails and browser cache of pentesters, which just mention a tool. ( linux ELF is uint16(0) == 0x457f, macos ( uint32be(0) == 0x7f454c46 or uint16(0) == 0xfeca or uint16(0) == 0xfacf or uint32(0) == 0xbebafeca ))

some repos for testing:

malware: https://github.com/Flangvik/SharpCollection https://github.com/tennc/webshell

goodware: ReactOS LiveCD: https://reactos.org/download/ any linux live DVD

sorry, that's a bunch of worky, but I think it's really needed to make this project usable.

best regards arnim