fedcode-next: Code pipeline and models to continuously collect scripts and vulnerability scanner rules to detect if the vulnerability is "expressed"

[pombredanne]

These are commonly published for important vulns. This can include yara rules, rules for IDS like Suricata, ClamAV entries, OpenVAS and nessus rules and related, like with MISP

[ziadhany]

Yara rules:

• https://github.com/elastic/protections-artifacts
• https://github.com/Yara-Rules/rules
• https://github.com/Xumeiquer/yara-forensics
• https://github.com/reversinglabs/reversinglabs-yara-rules
• https://github.com/advanced-threat-research/Yara-Rules
• https://github.com/bartblaze/Yara-rules
• https://github.com/godaddy/yara-rules
• https://github.com/SupportIntelligence/Icewater
• https://github.com/jeFF0Falltrades/YARA-Signatures
• https://github.com/tjnel/yara_repo
• https://github.com/JPCERTCC/jpcert-yara
• https://github.com/mikesxrs/Open-Source-YARA-rules
• https://github.com/fboldewin/YARA-rules
• https://github.com/h3x2b/yara-rules

Suricata rules:

• https://github.com/daffainfo/suricata-rules
• https://github.com/sudohyak/suricata-rules

Sigma rules:

• https://github.com/SigmaHQ/sigma

ClamAV:

• https://database.clamav.net/main.cvd
• https://database.clamav.net/daily.cvd
• https://database.clamav.net/bytecode.cvd

Snort Rule:

• https://github.com/thereisnotime/Snort-Rules

[ziadhany]

We have two options for storing rules:

• Pre-parsing: Parse the rule to extract specific fields before saving like (metadata , ....)
• Raw Storage: Store the raw text without validation.

Rule parsing :

• YARA

  • https://github.com/VirusTotal/yara-python
  • https://github.com/plyara/plyara
  • https://github.com/seifreed/yaraast

• Sigma Standard YAML files that are easy to parse.

• Suricata (No metadata)

  • https://github.com/m-chrome/py-suricataparser

• ClamAV (No metadata)

  • The signature base can be parsed using simple Python.

I think we can start with an initial model like this:

class DetectionRule(models.Model):
    RULE_TYPES = [
        ("yara", "YARA"),
        ("sigma", "Sigma Detection Rule"),
        ("clamav", "ClamAV Signature"),
        ("suricata", "Suricata Rule")
    ]

    rule_type = models.CharField(max_length=100, choices=RULE_TYPES, blank=True)

    source_url = models.URLField(
        null=True,
        blank=True,
    )

    rule_metadata = models.JSONField(
        null=True,
        blank=True,
    )
    rule_text = models.TextField()

    advisory = models.ForeignKey(
        AdvisoryV2,
        related_name="detection_rules",
        on_delete=models.SET_NULL,
        null=True,
        blank=True,
    )