[virustotal-livehunt-notifications]

[R3dHash]

Description

while using virustotal-livehunt-notifications connector a first bug is raised:

ERROR Object of type WhistleBlowerDict is not JSON serializable | timestamp=2024-02-17T21:54:50.926564Z name=VirusTotal Livehunt Notifications exc_info=Traceback (most recent call last):
  File "/opt/opencti-connector-virustotal-livehunt-notifications/livehunt/livehunt.py", line 289, in run
    self.builder.process(last_run, timestamp)
  File "/opt/opencti-connector-virustotal-livehunt-notifications/livehunt/builder.py", line 91, in process
    self.helper.log_debug(json.dumps(vtobj.__dict__, indent=2))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/encoder.py", line 202, in encode
    chunks = list(chunks)
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/encoder.py", line 432, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/local/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict
    yield from chunks
  File "/usr/local/lib/python3.11/json/encoder.py", line 439, in _iterencode
    o = _default(o)
        ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type WhistleBlowerDict is not JSON serializable

Environment

1. OS (where OpenCTI server runs): { e.g. Mac OS 10, Windows 10, Ubuntu 16.4, etc. }

Linux cti-super-downloader 6.1.0-16-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.67-1 (2023-12-12) x86_64 GNU/Linux

2. OpenCTI version: { e.g. OpenCTI 1.0.2 } 5.12.32
3. OpenCTI client: { e.g. frontend or python } Connector of VT using bundle STIX in the same docker-compose as Opencti to get activated
4. Other environment details:

connector-virustotal-livehunt-notifications:
    image: opencti/connector-virustotal-livehunt-notifications:${CONNECTOR_VERSION}
    environment:
      - OPENCTI_URL=http://opencti:8081
      - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
      - CONNECTOR_ID=ab9e0f18-8f40-4aae-ac28-49a3bfb0060f
      - CONNECTOR_TYPE=EXTERNAL_IMPORT
      - "CONNECTOR_NAME=VirusTotal Livehunt Notifications"
      - CONNECTOR_SCOPE=StixFile,Indicator,Incident
      - CONNECTOR_CONFIDENCE_LEVEL=50 # From 0 (Unknown) to 100 (Fully trusted)
      - CONNECTOR_LOG_LEVEL=error
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_API_KEY=${VTAPIKEY} # Private API Key
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_INTERVAL_SEC=60 # Time to wait in seconds between subsequent requests
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_CREATE_ALERT=False # Set to true to create alerts
      #- VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_EXTENSIONS='exe,dll' # (Optional) Comma separated filter to only download files matching these extensions
      #- VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_MIN_FILE_SIZE=1000 # (Optional) Don't download files smaller than this many bytes
      #- VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_MAX_FILE_SIZE=52428800 # (Optional) Don't download files larger than this many bytes
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_MAX_AGE_DAYS=30 # Only create the alert if the first submission of the file is not older than `max_age_days`
      #- VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_MIN_POSITIVES=5 # (Optional) Don't download files with less than this many vendors marking malicious
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_CREATE_FILE=False # Set to true to create file object linked to the alerts
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_UPLOAD_ARTIFACT=False # Set to true to upload the file to opencti
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_CREATE_YARA_RULE=False # Set to true to create yara rule linked to the alert and the file
      - VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_DELETE_NOTIFICATION=False # Set to true to remove livehunt notifications
      #- VIRUSTOTAL_LIVEHUNT_NOTIFICATIONS_FILTER_WITH_TAG='' # Filter livehunt notifications with this tag
  

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. { e.g. Run ... }
2. { e.g. Click ... }
3. { e.g. Error ... }

Expected Output

Actual Output

Additional information

Screenshots (optional)

Workaround

Whenever it tried with a previous version of the vt client (i.e., vt-py==0.18.0 by vt-py==0.17.5) it worked. But we had then another bug (not vt client related) linked to the fact that the connector originally works for files only. However, one can now chase also for network iocs like domains, IPs, etc. In that case the program raises an error at line 144 of the file src/livehunt/builder.py whenever a ruleset hunts for a nertwork ioc, which obviously has not a sha256 attribute.

# Create external reference to Virustotal report
            external_reference = self.create_external_reference(
                f"https://www.virustotal.com/gui/file/{vtobj.sha256}",
                "Virustotal Analysis",
            )

patched with this solves the problem in quick&dirty mode:

 if hasattr(vtobj,"sha256"):
     external_reference = self.create_external_reference(
                f"https://www.virustotal.com/gui/file/{vtobj.sha256}",
                "Virustotal Analysis",
            )

A cleaner way would be to get the typology of the ioc and craft the url to the vt report.

[nino-filigran]

@R3dHash do you have info about how to reproduce this bug? cc @helene-nguyen and @Megafredo @RaulSokolova, @axelfahy & @jtagcat I see that at some point you all have contributed to this connector. Would you mind having a look?

[R3dHash]

Hello, I guess you need a premium access to VT to make some tests. You should then enable a couple of dummy rulesets to produce a feed of IOCs that the connector should collect into opencti once you provide the api key related to your account. Then using the connector version 5.12.32 you should the first bug likely related to the client version should popup (see first screenshot). Then if you use yara rulesets with conditions like "vt.net.ip.ip_asn == XXXXX", as the results will be network ioc you'll get an error as no sha256 exists as attribute. Kind regards,

[AM-CERT]

Hello,

Just to give my workaround. Commenting out the line:

self.helper.log_debug(json.dumps(vtobj.__dict__, indent=2))

Resolved the issue for me.