Add integration tests to the developed Security Lake implementation

[mateojames]

Development branch
4055-aws-asl-integration-tests

Following the https://github.com/wazuh/wazuh/issues/16326 epic, this task aims to develop integrations tests to the https://github.com/wazuh/wazuh/issues/16407.

For the completion of this issue we must ensure to:

• [ ] Cover end-to-end testing of the integration.
• [ ] Consider edge cases and error scenarios to ensure the robustness of the integration.

[mateojames]

Issue Update

To do the testing I have a manager running Ubuntu in a Docker container, where the branch feature/16407-amazon-sec-lake-dev was cloned from the wazuh repository and installed. In the container I have also cloned the 3333-aws-integration-tests branch from wazuh-qa repository and have installed Vagrant for the management of VMs. Some errors were found.

🔴 Installation of the requirements error

root@wazuh-master:/home/wazuh-qa# python3 -m pip install -r requirements.txt
Ignoring pytest: markers 'python_version >= "3.10"' don't match your environment
Ignoring jq: markers '(platform_system == "Linux" or platform_system == "Darwin") and python_version >= "3.10"' don't match your environment
Ignoring wmi: markers 'platform_system == "Windows"' don't match your environment
Ignoring libcst: markers 'python_version <= "3.6"' don't match your environment
...
Successfully built configobj grpcio netifaces psutil pyyaml jq treelib pybitbucket-fork progress docopt ordered-set future
ERROR: grpcio-status 1.53.0 has requirement protobuf>=4.21.6, but you'll have protobuf 3.20.3 which is incompatible.
ERROR: python-daemon 3.0.1 has requirement setuptools>=62.4.0, but you'll have setuptools 56.0.0 which is incompatible.
Installing collected packages: six, configobj, certifi, pycparser, cffi, cycler, bcrypt, distro, filetype, python-dateutil, freezegun, grpcio, cachetools, pyasn1, rsa, pyasn1-modules, google-auth, urllib3, charset-normalizer, idna, requests, protobuf, googleapis-common-protos, grpcio-status, google-api-core, typing-extensions, pyyaml, mypy-extensions, typing-inspect, libcst, proto-plus, grpc-google-iam-v1, google-cloud-pubsub, setuptools, pyrsistent, attrs, jsonschema, kiwisolver, lockfile, pillow, numpy, contourpy, pyparsing, fonttools, zipp, importlib-resources, packaging, matplotlib, netifaces, pytz, tzdata, pandas, psutil, py, pycryptodome, cryptography, pyOpenSSL, iniconfig, pluggy, toml, pytest, pytest-metadata, pytest-html, scipy, seaborn, testinfra, jq, MarkupSafe, Jinja2, sphinxcontrib-devhelp, docutils, imagesize, sphinxcontrib-applehelp, importlib-metadata, babel, sphinxcontrib-htmlhelp, sphinxcontrib-qthelp, Pygments, snowballstemmer, sphinxcontrib-jsmath, sphinxcontrib-serializinghtml, alabaster, sphinx, numpydoc, python-daemon, ptyprocess, pexpect, ansible-runner, websocket-client, docker, python-vagrant, resolvelib, ansible-core, ansible, elastic-transport, elasticsearch, Click, dparse, safety, pbr, stevedore, smmap, gitdb, GitPython, bandit, future, oauthlib, requests-oauthlib, simplejson, uritemplate, voluptuous, pybitbucket-fork, uritemplate.py, github3.py, requests-toolbelt, python-gitlab, progress, docopt, gogs-client, lxml, git-repo, xmltodict, pyspnego, requests-ntlm, pywinrm, ordered-set, deepdiff, treelib, wcwidth, prettytable, mysql-connector-python
  Attempting uninstall: setuptools
    Found existing installation: setuptools 45.2.0
    Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'setuptools'. No files were found to uninstall.

🔴 Running a test to confirm a successful environment was deployed

------------------------------------------------------------------------------------------------- Captured log setup -------------------------------------------------------------------------------------------------
DEBUG    wazuh_testing:conftest.py:629 Set local_internal_option to {'wazuh_modules.debug': '2', 'monitord.rotate_log': '0'}
------------------------------------------------------------------------------------------------ Captured stderr call ------------------------------------------------------------------------------------------------
2023-04-17 19:12:36,016 - wazuh_testing - ERROR - ('The AWS module did not show the correct message about discard regex or ', 'did not process the expected amount of logs')
2023-04-17 19:12:36,018 - wazuh_testing - ERROR - Results accumulated: 0
2023-04-17 19:12:36,018 - wazuh_testing - ERROR - Results expected: 6
------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------
ERROR    wazuh_testing:monitoring.py:465 ('The AWS module did not show the correct message about discard regex or ', 'did not process the expected amount of logs')
ERROR    wazuh_testing:monitoring.py:466 Results accumulated: 0
ERROR    wazuh_testing:monitoring.py:468 Results expected: 6
---------------------------------------------------------------------------------------------- Captured stdout teardown ----------------------------------------------------------------------------------------------
wazuh-clusterd not running...
Killing wazuh-modulesd...
Killing wazuh-monitord...
Killing wazuh-logcollector...
Killing wazuh-remoted...
Killing wazuh-syscheckd...
Killing wazuh-analysisd...
wazuh-maild not running...
Killing wazuh-execd...
Killing wazuh-db...
Killing wazuh-authd...
wazuh-agentlessd not running...
wazuh-integratord not running...
wazuh-dbd not running...
wazuh-csyslogd not running...
Killing wazuh-apid...
Wazuh v4.4.1 Stopped
---------------------------------------------------------------------------------------------- Captured stderr teardown ----------------------------------------------------------------------------------------------
2023-04-17 19:12:38,903 - wazuh_testing - DEBUG - Restore local_internal_option to {}
----------------------------------------------------------------------------------------------- Captured log teardown ------------------------------------------------------------------------------------------------
DEBUG    wazuh_testing:conftest.py:634 Restore local_internal_option to {}
============================================================================================== short test summary info ===============================================================================================
FAILED test_aws/test_discard_regex.py::test_discard_regex[cloudtrail_discard_regex] - TimeoutError: ('The AWS module did not show the correct message about discard regex or ', 'did not process the expected amoun...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================================================== 1 failed, 1 passed, 180 deselected in 63.32s (0:01:03) ===============================================================================
root@wazuh-master:/home/wazuh-qa/tests/integration#

Further investigation is required.

[mateojames]

Amazon Security Lake Test Cases

Tier 0

• Amazon Security Lake integration works properly using the default configuration

  Pre-conditions

  • There are already configured credentials for the qa profile.
  • Debug mode is enabled in the internal_configuration file.

  Cases

  Cases	Expected Result	Configuration
  Use an existing lake and sqs	The module is invoked with the expected parameters and no error occurs	ossec.conf <wodle name="aws-s3"> <subscriber type="security_lake"> <sqs_name>ASL-Main-Queue</sqs_name> <iam_role_arn>arn:aws:iam::account-id:role/ASL-Role</iam_role_arn> <external_id>external-id</external_id> </subscriber> </wodle>

  Steps

  1. Apply the test case configuration to the ossec.conf
  2. Restart wazuh-manager service
  3. Monitor the ossec.log waiting until the wazuh service is restarted or there is a timeout
  4. Monitor the ossec.log waiting until the module is triggered or there is a timeout
  5. Check the command was called with the correct parameters
  6. Check no error messages are present in the ossec.log

• Amazon Security Lake integration works properly with iam_role_duration

  Pre-conditions

  • There are already configured credentials for the qa profile.
  • Debug mode is enabled in the internal_configuration file.

  Cases

  Cases	Expected Result	Configuration
  Use an existing lake and sqs	The module is invoked with the expected parameters and no error occurs	ossec.conf <wodle name="aws-s3"> <subscriber type="security_lake"> <sqs_name>ASL-Main-Queue</sqs_name> <iam_role_arn>arn:aws:iam::account-id:role/ASL-Role</iam_role_arn> <external_id>external-id</external_id> <iam_role_duration>role-duration</iam_role_duration> </subscriber> </wodle>

  Steps

  1. Apply the test case configuration to the ossec.conf
  2. Restart wazuh-manager service
  3. Monitor the ossec.log waiting until the wazuh service is restarted or there is a timeout
  4. Monitor the ossec.log waiting until the module is triggered or there is a timeout
  5. Check the command was called with the correct parameters
  6. Check no error messages are present in the ossec.log

• Amazon Security Lake integration works properly with sts_endpoint

  Pre-conditions

  • There are already configured credentials for the qa profile.
  • Debug mode is enabled in the internal_configuration file.

  Cases

  Cases	Expected Result	Configuration
  Use an existing lake and sqs	The module is invoked with the expected parameters and no error occurs	ossec.conf <wodle name="aws-s3"> <subscriber type="security_lake"> <sqs_name>ASL-Main-Queue</sqs_name> <iam_role_arn>arn:aws:iam::account-id:role/ASL-Role</iam_role_arn> <external_id>external-id</external_id> <sts_endpoint>sts-endpoint-IAM</sts_endpoint> </subscriber> </wodle>

  Steps

  1. Apply the test case configuration to the ossec.conf
  2. Restart wazuh-manager service
  3. Monitor the ossec.log waiting until the wazuh service is restarted or there is a timeout
  4. Monitor the ossec.log waiting until the module is triggered or there is a timeout
  5. Check the command was called with the correct parameters
  6. Check no error messages are present in the ossec.log

• Amazon Security Lake integration works properly with service_endpoint

  Pre-conditions

  • There are already configured credentials for the qa profile.
  • Debug mode is enabled in the internal_configuration file.

  Cases

  Cases	Expected Result	Configuration
  Use an existing lake and sqs	The module is invoked with the expected parameters and no error occurs	ossec.conf <wodle name="aws-s3"> <subscriber type="security_lake"> <sqs_name>ASL-Main-Queue</sqs_name> <iam_role_arn>arn:aws:iam::account-id:role/ASL-Role</iam_role_arn> <external_id>external-id</external_id> <service_endpoint>sqs.region.amazonaws.com</service_endpoint> </subscriber> </wodle>

  Steps

  1. Apply the test case configuration to the ossec.conf
  2. Restart wazuh-manager service
  3. Monitor the ossec.log waiting until the wazuh service is restarted or there is a timeout
  4. Monitor the ossec.log waiting until the module is triggered or there is a timeout
  5. Check the command was called with the correct parameters
  6. Check no error messages are present in the ossec.log

[mateojames]

Issue Update

Some integration tests still need to be defined, such as:

• Verify that when a subscriber section has more than one security lake section, the module works correctly
• Verify that the duration of the role works only for the defined time (iam_role_duration)
• Verify that the sts_endpoint is implemented correctly (procedure to be defined)
• Verify that the service_endpoint is implemented correctly (procedure to be defined)
• Verify that the queue messages are deleted correctly after being read

[mateojames]

Issue Update

The subscriber section for Tier 0 testing was added to the test_basic, still in development process

[fdalmaup]

Issue Update

To maintain consistency between the different already developed integration tests of the module, and because the iam_role_duration, sts_endpoint, and service_endpoint parameters do not generate a fundamental modification of the behavior of the module, no tests will be developed that make use of the same.

Also, the parser integration tests added in https://github.com/wazuh/wazuh-qa/pull/3882 will be expanded in order to contemplate the new <subscriber> section and its inner fields.

[fdalmaup]

Issue Update

The current Integration Tests structure has been reviewed to understand better how the Security Lake tests should be developed.

Since the objective of these tests is not the generation of the Security Lake itself but the interaction with the principal components available when setting up a Data Access Subscription, we should have an environment with the following services available for the test account:

• IAM: a Role with the necessary permissions to access the SQS and S3 services. It will be the one to be assumed by the module with the iam_role_arn parameter when fetching messages and the bucket's content and it will also have the external_id parameter set in Trusted relationships.
• SQS: a Queue that simulates the one created by the Subscription and required for the sqs_name parameter. The test will send a message to it which will be the one ingested by the module that contains the path of the parquet file.
• S3: a bucket with a sample parquet file located in the corresponding path.

While the needed permissions to handle these services are being required, the Subscriber Parser Integration tests are being developed.

[fdalmaup]

Issue Update

The base test which needs the previously mentioned AWS services, will be added to test_basic.py.

In the meantime, the test_parser.py is being updated with ASL integration cases, which are:

• [x] test_parser.py::test_type_missing_in_subscriber[parser_type_missing_in_subscriber]
• [x] test_parser.py::test_empty_values_in_subscriber[parser_empty_type_in_subscriber]
• [ ] test_parser.py::test_empty_values_in_subscriber[parser_empty_queue_in_subscriber] The expected message related to the empty sqs_name value is not implemented at the C level. Instead, it is being processed by the script as an invalid argument due to its non-existence: WARNING: Subscriber: security_lake - Error parsing arguments.

Points to discuss: Should we take it as an invalid value or a required one like the subscriber type which does not allow the empty value? This last option would require an issue in order to carry out the necessary changes (should be opened after the whole analysis on the required changes)

• [ ] test_parser.py::test_invalid_values_in_subscriber[parser_invalid_type_in_subscriber]

• [ ] test_parser.py::test_invalid_values_in_subscriber[parser_invalid_queue_in_subscriber]

• [ ] test_parser.py::test_multiple_bucket_and_service_tags[parser_mutiple_bucket_and_service_tags]

Possible alternatives: add more test cases or add the subscriber configuration to the existing one.

[fdalmaup]

Issue Update

New test cases have been checked for the Parser tests:

Developed with the current implementation:

• test_parser.py::test_type_missing_in_subscriber[parser_type_missing_in_subscriber]
• test_parser.py::test_empty_values_in_subscriber[parser_empty_type_in_subscriber]
• test_parser.py::test_invalid_values_in_subscriber[parser_invalid_type_in_subscriber]
• test_parser.py::test_multiple_bucket_service_and_subscriber_tags[parser_multiple_bucket_service_and_subscriber_tags]

May require modifications of the module at a C and/or script level

• test_parser.py::test_empty_values_in_subscriber[parser_empty_queue_in_subscriber] Displays: WARNING: Subscriber: security_lake - Error parsing arguments. The sqs_name is a mandatory parameter, the module should show an error when it parses the ossec.conf and does not find a value for the tag.

• test_parser.py::test_empty_values_in_subscriber[parser_empty_iam_arn_in_subscriber] Raises an error at a script level, displaying the traceback. Needs to be modified because the iam_role_arn is required for the subscriber type.

• test_parser.py::test_empty_values_in_subscriber[parser_empty_external_id_in_subscriber] It also raises an error at a script level, displaying the traceback. Needs to be modified because the external_id is required for the subscriber type.

• test_parser.py::test_invalid_values_in_subscriber[parser_invalid_queue_in_subscriber] This is not currently being checked, so the error displayed is the one when no queue with the given name is found. AWS sets minimal requirements for these, therefore the argument verification should be implemented.

• test_parser.py::test_invalid_values_in_subscriber[parser_invalid_iam_arn_in_subscriber] This also is not currently being checked and given that the iam_role_arn is a mandatory parameter, a verification function should be developed.

• test_parser.py::test_invalid_values_in_subscriber[parser_invalid_iam_duration_in_subscriber] It should be modified with a more descriptive message because when setting a non-alphanumeric value, the following logs are displayed:

2023/06/22 14:47:56 wazuh-modulesd:aws-s3[39534] wm_aws.c:790 at wm_aws_run_subscriber(): DEBUG: Launching Security Lake Subscriber Command: wodles/aws/aws-s3 --subscriber security_lake --queue name --external_id invented --iam_role_arn role --iam_role_duration test --debug 2
2023/06/22 14:47:56 wazuh-modulesd:aws-s3[39534] wm_aws.c:805 at wm_aws_run_subscriber(): WARNING: Subscriber: security_lake name  -  Returned exit code 2
2023/06/22 14:47:56 wazuh-modulesd:aws-s3[39534] wm_aws.c:818 at wm_aws_run_subscriber(): WARNING: Subscriber: security_lake name  -  Error parsing arguments.

[Selutario]

Moving to on hold as the tests migration is almost ready.

[fdalmaup]

The issue will be resumed once higher-priority issues are done.

[fdalmaup]

Due to the changes planned for version 5.0, the analysis that is being carried out regarding the External integrations modules, the team's efforts for improvements will be directed to the mentioned version.