wazuh
wazuh copied to clipboard
Published
20 hours ago •
wazuh
wazuh
Upgrade cti to connexion 3.0
| Wazuh version | Component |
|---|---|
| 4.9 | api, framework |
Description
The original cti endpoint was developed with the aiohttp package. It must be updated to use the connexion 3.0 package:
- All imports to aiohttp and related dependencies must be replaced.
- The
wazuh-apidmust be modified to work withuvicornserver. - The
manager_controllers.test_check_available_versionmust be upgraded to the new version. - The
wazuh.core.manager.query_update_check_servicefunction must be modified to avoidaiohttpdependencies.
Checks
The following elements have been updated or reviewed (should also be checked if no modification is required):
- [ ] Tests (unit tests, API integration tests).
- [ ] Changelog.
- [ ] Documentation.
- [ ] Integration test mapping (using
api/test/integration/mapping/_test_mapping.py).
Update
I've been talking with @mhamra and we decided to migrate the background tasks, of the CTI functionality, to the middleware:
https://github.com/wazuh/wazuh/blob/8aefebc04f0021450748cfea36246893f37666dc/api/api/middlewares.py#L264-L271
So, the modifications need to be made in this direction.
Update
I've been working in the lifespan logic for migrating the background tasks but, I got stuck by the lack of a global object to share the information between the tasks and the requests.
I need to check some workarounds because the context object, that Connexion 3 provides, isn't accessible outside the request/response cycle.
Update
I've been making progress in implementing the background tasks and fixing the unit test of the framework and the API.
root@wazuh-master:/var/ossec# ./api/scripts/wazuh-apid.py -r -f
2024/02/29 21:08:09 INFO: Starting API in foreground
2024/02/29 21:08:09 INFO: Starting API as root
2024/02/29 21:08:09 INFO: Checking RBAC database integrity...
2024/02/29 21:08:09 INFO: /var/ossec/api/configuration/security/rbac.db file was detected
2024/02/29 21:08:09 INFO: RBAC database integrity check finished successfully
2024/02/29 21:08:13 INFO: Listening on 0.0.0.0:55000.
2024/02/29 21:08:13 INFO: Getting installation UID...
2024/02/29 21:08:13 INFO: Getting updates information...
2024/02/29 21:08:22 INFO: wazuh 172.26.0.1 "GET /security/user/authenticate" with parameters {"raw": "true"} and body {} done in 0.249s: 200
2024/02/29 21:08:26 ERROR: Error in CTI service request: 07148b46-075e-47f3-b883-ee2545bd45c2, v4.9.0
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/var/ossec/framework/python/lib/python3.10/concurrent/futures/process.py", line 246, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/var/ossec/framework/python/lib/python3.10/site-packages/wazuh/core/cluster/dapi/dapi.py", line 239, in run_local
data = f(**f_kwargs)
File "/var/ossec/framework/python/lib/python3.10/site-packages/wazuh/manager.py", line 421, in get_update_information
raise WazuhInternalError(2100, extra_message=extra_message)
wazuh.core.exception.WazuhInternalError: Error 2100 - Error in CTI service request: 07148b46-075e-47f3-b883-ee2545bd45c2, v4.9.0
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/ossec/framework/python/lib/python3.10/site-packages/wazuh/core/cluster/dapi/dapi.py", line 285, in execute_local_request
data = await asyncio.wait_for(task, timeout=timeout)
File "/var/ossec/framework/python/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
wazuh.core.exception.WazuhInternalError: Error 2100 - Error in CTI service request: 07148b46-075e-47f3-b883-ee2545bd45c2, v4.9.0
2024/02/29 21:08:26 INFO: wazuh 172.26.0.1 "GET /manager/version/check" with parameters {} and body {} done in 0.044s: 500
➜ ~ curl -k --silent -H "Authorization: Bearer $TOKEN" https://localhost:55050/manager/version/check | jq
{
"title": "Wazuh Internal Error",
"detail": "Error in CTI service request: 07148b46-075e-47f3-b883-ee2545bd45c2, v4.9.0",
"type": "about:blank",
"dapi_errors": {
"master-node": {
"error": "Error in CTI service request: 07148b46-075e-47f3-b883-ee2545bd45c2, v4.9.0",
"logfile": "WAZUH_HOME/logs/api.log"
}
},
"error": 2100
}
UT
(framework) ➜ wazuh git:(feature/21651-upgrade-cti-to-connexion-30) ✗ PYTHONPATH=$WAZUH_REPO/api:$WAZUH_REPO/framework python -m pytest --disable-warnings framework
============================= test session starts ==============================
platform linux -- Python 3.10.13, pytest-7.3.1, pluggy-1.4.0
rootdir: /home/nstefani/git/wazuh/framework
configfile: pytest.ini
plugins: asyncio-0.18.1, tavern-1.23.5, trio-0.8.0, html-2.1.1, metadata-3.1.1, anyio-4.3.0
asyncio: mode=auto
collected 2211 items
framework/scripts/tests/test_agent_groups.py .............. [ 0%]
framework/scripts/tests/test_agent_upgrade.py ............... [ 1%]
framework/scripts/tests/test_cluster_control.py ...... [ 1%]
framework/scripts/tests/test_rbac_control.py ......... [ 1%]
framework/scripts/tests/test_wazuh_clusterd.py ....... [ 2%]
framework/scripts/tests/test_wazuh_logtest.py ...................... [ 3%]
framework/wazuh/core/cluster/dapi/tests/test_dapi.py ................... [ 4%]
............. [ 4%]
framework/wazuh/core/cluster/tests/test_client.py ................ [ 5%]
framework/wazuh/core/cluster/tests/test_cluster.py ..................... [ 6%]
.............. [ 7%]
framework/wazuh/core/cluster/tests/test_common.py ...................... [ 8%]
.............................................................. [ 10%]
framework/wazuh/core/cluster/tests/test_control.py ...... [ 11%]
framework/wazuh/core/cluster/tests/test_local_client.py .............. [ 11%]
framework/wazuh/core/cluster/tests/test_local_server.py ................ [ 12%]
........ [ 12%]
framework/wazuh/core/cluster/tests/test_master.py ...................... [ 13%]
......................... [ 14%]
framework/wazuh/core/cluster/tests/test_server.py ...................... [ 15%]
....... [ 16%]
framework/wazuh/core/cluster/tests/test_utils.py ................ [ 17%]
framework/wazuh/core/cluster/tests/test_worker.py ...................... [ 18%]
............ [ 18%]
framework/wazuh/core/tests/test_active_response.py .................... [ 19%]
framework/wazuh/core/tests/test_agent.py ............................... [ 20%]
........................................................................ [ 24%]
.............................................. [ 26%]
framework/wazuh/core/tests/test_cdb_list.py ............................ [ 27%]
.......... [ 27%]
framework/wazuh/core/tests/test_common.py ......... [ 28%]
framework/wazuh/core/tests/test_configuration.py ....................... [ 29%]
.................................................................. [ 32%]
framework/wazuh/core/tests/test_decoder.py ................ [ 33%]
framework/wazuh/core/tests/test_exception.py .......... [ 33%]
framework/wazuh/core/tests/test_input_validator.py ... [ 33%]
framework/wazuh/core/tests/test_logtest.py ... [ 33%]
framework/wazuh/core/tests/test_manager.py ............................. [ 35%]
.. [ 35%]
framework/wazuh/core/tests/test_mitre.py ............. [ 35%]
framework/wazuh/core/tests/test_pyDaemonModule.py ..... [ 36%]
framework/wazuh/core/tests/test_results.py ............................. [ 37%]
........... [ 37%]
framework/wazuh/core/tests/test_rootcheck.py ............. [ 38%]
framework/wazuh/core/tests/test_rule.py ....................... [ 39%]
framework/wazuh/core/tests/test_sca.py ............................. [ 40%]
framework/wazuh/core/tests/test_security.py ............. [ 41%]
framework/wazuh/core/tests/test_stats.py ............................... [ 42%]
......... [ 43%]
framework/wazuh/core/tests/test_syscheck.py ....... [ 43%]
framework/wazuh/core/tests/test_syscollector.py ... [ 43%]
framework/wazuh/core/tests/test_task.py ........ [ 43%]
framework/wazuh/core/tests/test_utils.py ............................... [ 45%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 55%]
................................. [ 56%]
framework/wazuh/core/tests/test_wazuh_queue.py ....................... [ 57%]
framework/wazuh/core/tests/test_wazuh_socket.py .................... [ 58%]
framework/wazuh/core/tests/test_wdb.py ............................... [ 59%]
framework/wazuh/core/tests/test_wlogging.py ............ [ 60%]
framework/wazuh/rbac/tests/test_auth_context.py .. [ 60%]
framework/wazuh/rbac/tests/test_decorators.py .......................... [ 61%]
........................................................................ [ 65%]
........... [ 65%]
framework/wazuh/rbac/tests/test_default_configuration.py ............... [ 66%]
........................................ [ 68%]
framework/wazuh/rbac/tests/test_orm.py ................................. [ 69%]
............................. [ 70%]
framework/wazuh/rbac/tests/test_preprocessor.py ........... [ 71%]
framework/wazuh/tests/test_active_response.py ............ [ 71%]
framework/wazuh/tests/test_agent.py .................................... [ 73%]
........................................................................ [ 76%]
..................... [ 77%]
framework/wazuh/tests/test_cdb_list.py ................................. [ 79%]
.................... [ 80%]
framework/wazuh/tests/test_ciscat.py ................................. [ 81%]
framework/wazuh/tests/test_cluster.py .......... [ 82%]
framework/wazuh/tests/test_decoder.py .................................. [ 83%]
........................ [ 84%]
framework/wazuh/tests/test_event.py .... [ 84%]
framework/wazuh/tests/test_logtest.py ...... [ 85%]
framework/wazuh/tests/test_manager.py .................................. [ 86%]
.. [ 86%]
framework/wazuh/tests/test_mitre.py ....... [ 87%]
framework/wazuh/tests/test_rootcheck.py ................................ [ 88%]
.................. [ 89%]
framework/wazuh/tests/test_rule.py ..................................... [ 90%]
..................................... [ 92%]
framework/wazuh/tests/test_sca.py ........... [ 93%]
framework/wazuh/tests/test_security.py ................................. [ 94%]
...................................... [ 96%]
framework/wazuh/tests/test_stats.py ............... [ 97%]
framework/wazuh/tests/test_syscheck.py ......................... [ 98%]
framework/wazuh/tests/test_syscollector.py ............ [ 98%]
framework/wazuh/tests/test_task.py ............................ [100%]
================ 2211 passed, 10 warnings in 285.22s (0:04:45) =================
(framework) ➜ wazuh git:(feature/21651-upgrade-cti-to-connexion-30) ✗ PYTHONPATH=$WAZUH_REPO/api:$WAZUH_REPO/framework python -m pytest --disable-warnings api/api
============================= test session starts ==============================
platform linux -- Python 3.10.13, pytest-7.3.1, pluggy-1.4.0
rootdir: /home/nstefani/git/wazuh/api/api
configfile: pytest.ini
plugins: asyncio-0.18.1, tavern-1.23.5, trio-0.8.0, html-2.1.1, metadata-3.1.1, anyio-4.3.0
asyncio: mode=auto
collected 603 items
api/api/controllers/test/test_active_response_controller.py . [ 0%]
api/api/controllers/test/test_agent_controller.py ...................... [ 3%]
..................... [ 7%]
api/api/controllers/test/test_cdb_list_controller.py ...... [ 8%]
api/api/controllers/test/test_ciscat_controller.py . [ 8%]
api/api/controllers/test/test_cluster_controller.py .................... [ 11%]
.... [ 12%]
api/api/controllers/test/test_controller_util.py .. [ 12%]
api/api/controllers/test/test_decoder_controller.py ....... [ 13%]
api/api/controllers/test/test_default_controller.py . [ 14%]
api/api/controllers/test/test_event_controller.py . [ 14%]
api/api/controllers/test/test_experimental_controller.py ............... [ 16%]
[ 16%]
api/api/controllers/test/test_manager_controller.py .................... [ 20%]
. [ 20%]
api/api/controllers/test/test_mitre_controller.py ....... [ 21%]
api/api/controllers/test/test_overview_controller.py . [ 21%]
api/api/controllers/test/test_rootcheck_controller.py .... [ 22%]
api/api/controllers/test/test_rule_controller.py ........ [ 23%]
api/api/controllers/test/test_sca_controller.py .. [ 23%]
api/api/controllers/test/test_security_controller.py ................... [ 27%]
................................ [ 32%]
api/api/controllers/test/test_syscheck_controller.py .... [ 33%]
api/api/controllers/test/test_syscollector_controller.py ......... [ 34%]
api/api/controllers/test/test_task_controller.py . [ 34%]
api/api/models/test/test_model.py .............................. [ 39%]
api/api/test/test_alogging.py ............. [ 41%]
api/api/test/test_authentication.py ........... [ 43%]
api/api/test/test_configuration.py ..................................... [ 49%]
........ [ 51%]
api/api/test/test_encoder.py ... [ 51%]
api/api/test/test_error_handler.py ............................. [ 56%]
api/api/test/test_middlewares.py ........................... [ 60%]
api/api/test/test_signals.py ......... [ 62%]
api/api/test/test_uri_parser.py .......... [ 64%]
api/api/test/test_util.py ......................................... [ 70%]
api/api/test/test_validator.py ......................................... [ 77%]
........................................................................ [ 89%]
............................................................... [100%]
======================= 603 passed, 75 warnings in 6.78s =======================