AttributeError: 'IntegrationTest' object has no attribute 'attack_data'

[Res260]

Hi, when running contentctl build --verbose, I get this weird error:

C:\Users\user\.virtualenvs\something\Scripts\python.exe -m contentctl.contentctl build --verbose 
INFO: Common Information Model/CIM (uid: [1621]) is not listed in apps.
contentctl test MUST include Common Information Model.
Please note this message is only informational.
INFO: Common Information Model/CIM (uid: [1621]) is not listed in apps.
contentctl test MUST include Common Information Model.
Please note this message is only informational.
   DEPLOYMENTS Progress: [100%]...Done!
       LOOKUPS Progress: [  0%]...Done!
        MACROS Progress: [100%]...Done!
       STORIES Progress: [100%]...Done!
     BASELINES Progress: [  0%]...Done!
INVESTIGATIONS Progress: [  0%]...Done!
  DATA_SOURCES Progress: [100%]...Done!
     PLAYBOOKS Progress: [  0%]...Done!
Verbose error logging is ENABLED.
The entire stack trace has been provided below (please include it if filing a bug report):

Traceback (most recent call last):
  File "C:\something\contentctl\contentctl\contentctl.py", line 203, in main
    build_func(config)
  File "C:\something\contentctl\contentctl\contentctl.py", line 78, in build_func
    director_output_dto = validate_func(config)
                          ^^^^^^^^^^^^^^^^^^^^^
  File "C:\something\contentctl\contentctl\contentctl.py", line 63, in validate_func
    return validate.execute(config)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\something\contentctl\contentctl\actions\validate.py", line 37, in execute
    director.execute(input_dto)
  File "C:\something\contentctl\contentctl\input\director.py", line 131, in execute
    self.createSecurityContent(SecurityContentType.detections)
  File "C:\something\contentctl\contentctl\input\director.py", line 210, in createSecurityContent
    detection = Detection.model_validate(modelDict,context={"output_dto":self.output_dto, "app":self.input_dto.app})
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\.virtualenvs\something\Lib\site-packages\pydantic\main.py", line 551, in model_validate
    return cls.__pydantic_validator__.validate_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\something\contentctl\contentctl\objects\abstract_security_content_objects\detection_abstract.py", line 147, in validate_test_groups
    test_group = TestGroup.derive_from_unit_test(unit_test, info.data.get("name"))          # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\something\contentctl\contentctl\objects\test_group.py", line 39, in derive_from_unit_test
    attack_data=unit_test.attack_data
                ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\.virtualenvs\something\Lib\site-packages\pydantic\main.py", line 811, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'IntegrationTest' object has no attribute 'attack_data'

Process finished with exit code 1

Example detection yaml:

author: someone
data_sources: []
date: '2024-08-14'
description: "something"
how_to_implement: something
id: 8e7eeb73-69f4-48cf-b1f4-aaaaaaaaaaaa
known_false_positives: something
name: some detection
references: []
search: "whatever"
status: production
tags:
  analytic_story: []
  asset_type: Account
  confidence: 1
  cve: []
  impact: 60
  message: N/A
  mitre_attack_id:
  - T1555
  observable:
  - name: ' '
    role:
    - Victim
    type: Unknown
  product:
  - Splunk Enterprise Security
  required_fields:
  - nothing
  risk_score: N/A
  security_domain: access
tests:
- attack_data:
  - data: ./something
    source: N/A
    sourcetype: N/A
  name: somethingelse
  test_type: unit
type: TTP
version: 1

There seems to be a problem with the parsing in Detection_Abstract.validate_test_groups(). It looks like a logic error, where the code assumes it’s a UnitTest object but receives a IntegrationTest object.

Any idea?

[ljstella]

Is that example the exact structure you've got in your YAML? (Keys, not values)

If you don't want to use testing at all, you can set it to:

tests:
- name: Null Test
  attack_data: []

If you're creating a new detection with contentctl new --type detection you'll get something like this:

tests:
- name: True Positive Test
  attack_data:
  - data: https://github.com/splunk/contentctl/wiki
    sourcetype: UPDATE SOURCETYPE
    source: UPDATE SOURCE

which does pass the validation.

The issue in your specific example, which is likely the cause of the issue in your actual detection is the - character next to attack_data. If you look at the above example, the structure should be

tests:
- name: Name of Test
  attack_data:
  - data: https://link-to-data
    sourcetype: sourcetype:of:data
    source: source-of.log

Because the YAML is misshappen, it loads it differently, hence the complaining about the integration tests.

[Res260]

Hello, thanks for your reply.

This is not the cause of the problem.

tests:
- attack_data:
  - data: ./something
    source: N/A
    sourcetype: N/A
  name: somethingelse
  test_type: unit

is the same as

tests:
- name: somethingelse
  attack_data:
  - data: ./something
    source: N/A
    sourcetype: N/A
  test_type: unit

I tried both and got the same error both times.

However, I found that this does not create the error, which is a very weird behavior

tests:
- attack_data:
  - data: https://something.com/something
    source: N/A
    sourcetype: N/A
  name: somethingelse
  test_type: unit

Notice the link in the data key instead of a file path. From my understanding, file paths should be accepted as a correct value. Am I missing something?

I am trying to make tests work with our custom detection-as-code pipeline, because the "test using docker" feature of contentctl would be of major help to us :)

Thanks for your help.