wazuh-packages icon indicating copy to clipboard operation
wazuh-packages copied to clipboard

Published 20 hours ago • verified publisher icon wazuh

Wazuh packages redesign tier 2

Open havidarou opened this issue 10 months ago • 1 comments

Description

Wazuh-packages (QA, Agent, CppServer)

  • Clean up everything
  • Add support to PPC packages (Agent)

Requirements and Restrictions

  • [x] Wazuh packages redesign tier 1 requirements should be taken into account
  • [ ] The new repository must include QA related code and tools
    • [ ] New Jenkins should be used for everything QA related after 5.0
    • [ ] Current wazuh-packages repository is deprecated after 4.10
    • [ ] Current wazuh-jenkins repository is deprecated after 4.10
    • [x] Current wazuh-tools repository is deprecated after 4.10
    • [x] Migrate scripts and pipelines that sync GitHub projects with GitHub.(QA)
    • [x] Identify existing pipelines and migrate those that continue to provide coverage to Wazuh 4.x. (QA)
  • [x] The release procedure should be improved by: - Improving repository metadata generation (avoid full metadata sync) - Reducing execution time - Incorporating backups and storage - Mirroring production bucket (release to the mirror, sync main with the mirror, restore main with the mirror)
  • [x] The following will be migrated to the new DevOps repositories (both packages and automation): - OVA - AMI - Wazuh unattended scripts - Wazuh Puppet - Wazuh Docker - Wazuh Kubernetes - Wazuh Ansible - Training environment - Demo environment
  • [x] Add support to PPC packages (Agent)
  • [x] The following installers will be migrated from wazuh-packages repository: (4.10.1) Agent installers: - HP-UX - Solaris Intel/SPARC - AIX

Auditor

  • Involved: DevOps, QA, Agent, CppServer
  • esponsible: @wazuh/devel-qa-div3
  • DRI name: @davidjiglesias
  • Objective: Wazuh packages redesign tier 2

Planning

Precondition

Previous tier

  • https://github.com/wazuh/internal-devel-requests/issues/187

Dependencies

  • https://github.com/wazuh/wazuh-jenkins/issues/5927
  • https://github.com/wazuh/wazuh-jenkins/issues/6357
  • https://github.com/wazuh/wazuh-jenkins/issues/6471
Spike
  • https://github.com/wazuh/wazuh-qa/issues/5527 (QA)
  • https://github.com/wazuh/wazuh-qa/issues/5574 (QA)
  • https://github.com/wazuh/wazuh-qa/issues/5599
Tasks

QA

  • https://github.com/wazuh/wazuh-jenkins/issues/6811
  • https://github.com/wazuh/wazuh-jenkins/issues/6770
  • https://github.com/wazuh/wazuh-qa-automation/issues/56
  • https://github.com/wazuh/wazuh-qa-automation/issues/26
  • https://github.com/wazuh/internal-devel-requests/issues/1558
  • https://github.com/wazuh/internal-devel-requests/issues/1555
  • https://github.com/wazuh/wazuh-qa-automation/issues/113
  • https://github.com/wazuh/wazuh-qa-automation/issues/111
  • https://github.com/wazuh/wazuh-qa-automation/issues/160
  • https://github.com/wazuh/wazuh-qa/issues/5556
  • https://github.com/wazuh/wazuh-qa-automation/issues/130
  • https://github.com/wazuh/wazuh-qa-automation/issues/84

Further improvements are required

  • https://github.com/wazuh/wazuh-qa-automation/issues/221
  • https://github.com/wazuh/wazuh-qa-automation/issues/226

DevOps

  • https://github.com/wazuh/wazuh-automation/issues/1658
  • https://github.com/wazuh/wazuh-virtual-machines/issues/22
  • https://github.com/wazuh/wazuh-installation-assistant/issues/21
  • https://github.com/wazuh/wazuh-automation/issues/1766
  • https://github.com/wazuh/wazuh-automation/issues/1767
  • https://github.com/wazuh/wazuh-qa/issues/5654
  • https://github.com/wazuh/wazuh-automation/issues/1873

Agent

  • https://github.com/wazuh/wazuh-agent-packages/issues/55
  • https://github.com/wazuh/wazuh/issues/26490
  • https://github.com/wazuh/wazuh/issues/25683

Conclusion (WIP)

QA

Release procedure

  • S3 sign check: S3 sign checking now works in multi-threaded mode, where each thread runs in a separate VM to handle its errors (destroying the VM and leaving a log of the error). It helps achieve asynchronous independence from the executions in S3_sign_check.
  • Reducing execution time: There's no way to speed up DEB metadata generation without creating bigger problems that could compromise the process.
  • Allow release-tool.py execution on AWS: It allows the launch tool to run on AWS infrastructure (4vcpu and 8GB). In addition, it was adapted to local execution and in local containers, providing a single entry point for the tool. It balanced the increase in time in metadata generation.
    Stage v4.9.0-beta2 Current implementation on AWS
    RPM - Download 52m 7m
    RPM - Metadata and upload - 4m20s
    RPM - Metadata 20s -
    RPM - Upload 40s -
    DEB - Download 49m 5m
    DEB - Metadata and upload - 46m
    DEB - Metadata 22m -
    DEB - Upload 40s -
  • Improving repository metadata generation: Added new documentation on how to modify recipe files and regenerate metadata without adding files to the repository, added new use cases such as Stage support, release protocol, and metadata regeneration, improved requirements, and removed redundant information.
  • Incorporating backups and storage: It provides an optional backup mechanism and a precondition feature that allows the execution of ordered tasks and allows some operations before starting with the core release procedure.
  • Mirroring production bucket (release to the mirror, sync main with the mirror, restore main with the mirror): Currently, we sync prod into prod-mirror to guarantee that prod-mirror is a mirror and deploy packages into prod-mirror. After that, the backup prod by into the directory (s3://packages.wazuh.com/Backups/Backup_{timestamp}/{major}.x) and we sync prod-mirror into prod. Finally, we AWS S3-to-S3 copy, without localhost intervention, only deltas, and then we invalidate the cache of the prod.

Restore support to PPC packages (Agent - deb ppc64el, rpm ppc64le)

The changes consisted of:

  • Update Wazuh documentation.
  • Adapt release procedures to support PPC packages (package generation, package signing and verification, and release tool deployment).
  • Update release issues (tests).

The new repository must include QA related code and tools

  • Defines a private repository: “wazuh-qa-automation”.
  • Currently, we have two branches: 4.10.0 and the main.
  • The new repository includes:
    • New templates for reporting bugs, requesting new support, and creating PRs.
    • New README with a use guide for the tools migrated.
    • Apply new structure and good practices.
    • The 4.10.0 and 5.0.0 branches have the same migrated code. It contains:
      • Workflow
        • s3_sign_check: Adds the requirement for a GitHub PAT that must be set using the GITHUB_TOKEN environment variable.
        • dtt1: Adds two steps to workflow files. Now, the token must be manually filled in before execution.
      • Provisioning tool
      • Release procedures
      • The new Jenkins
      • Testing tool
      • SCA footprint test (sca_footprint)
      • System tests
      • VD E2E tests
      • Test_upgrade
      • Test_upgrade_tier
      • Test_stress
      • Test_stress_gh
      • Test_stress_tier
      • Test_stress_Vagrant
      • Procedure_ecs_task
      • Procedure_ecs_task_tier
      • Procedure_gh_project_syncup
      • procedure_cicd_release_issues
OUT OF SCOPE: "Packages Redesign Tier 2"

  • We are working on https://github.com/wazuh/wazuh-qa-automation/issues/42 which proposes new processes and practices to work.

  • The 5.0.0 branch should remove everything that Migrate has deprecated.

  • Once the target tasks are complete, we will work on implementing some improvements, which are:

    Release Procedures

    Medium priority improvements:

    • tools/release/src/**/docker/Dockerfile_tests: Each module's docker test image. We should re-evaluate if it's necessary to run each module's unit tests on a docker image, noting that it consumes time and may not add much value to the development itself.
    • .github/workflows: Main GHA workflows directory
      • Currently, all the Release Procedures modules' Unit Tests are being executed on GHA and running on Docker, which means that we build a Docker image and run it on each GHA, and it is resources-consuming.
      • All the GHA Workflows can be simplified in just one workflows that runs all the UTs (but first, it would be great if we get rid of the Docker build)
    Testing

    High-priority improvements:

    • tools/testing/src/testing/testing.py: Main testing module file
      • The module's parameters are strongly attached to DTT, it must be more generic to re-use it on different suites.
      • It expects the target and dependencies inventories to be on a specific path with a desired format, so it won't detect the target system correctly if the inventory is in a custom directory.
    • tools/testing/src/testing/playbooks/test.yml Test execution playbook The test path is generated "dynamically" but it depends completely on the DTT format
    • tests/test_functional/test_system/test_deployability/workflows: DTT workflows directory
      • The workflows are outdated, and its using the legacy allocator path, it must be updated to use them.
      • Wazuh version on the Workflows is hardcoded using an outdated value.
      • Add parametrization on the Workflows where possible.
    • tools/testing/tests: Testing tool Unit Tests directory There are no Unit Tests developed, we must create a set of tests to ensure its correct functionality Related issues:
      • https://github.com/wazuh/wazuh-qa/issues/4993

    Medium priority improvements:

    • tests/test_functional/test_system/test_deployability/tests: Main DTT tests directory The test helpers have a lot of duplicated code, and there are several wrongly handled conditional flows. We could include unit tests for the helpers to validate its correct behavior
    JobFlow

    Medium priority improvements:

    • tools/jobflow/README.md: Main documentation file of the JobFlow tool Currently, the documentation is DTT1-specific, it does not help the user to understand what the tool does, and how it can be used for different cases.

    Low priority improvements:

    • tools/jobflow/tests: Unit Tests suite This suite uses Python's default test runner UnitTests utilities and pytest as the runner, it could be improved by only using pytest and its utilities, which is recommended. Related issues:
      • https://github.com/wazuh/wazuh-qa/issues/4993
    • tools/jobflow/examples: Workflows examples directory We must add more Workflows as examples of different use cases.

    Other improvements could be found here: https://github.com/wazuh/wazuh-qa/issues/5044

    Provision

    Medium priority improvements:

    • tools/provision/README.md: Main documentation file of the Provision tool This README is strongly related to the JobFlow, it could be improved detailing mainly the standalone execution and having the JobFlow workflow as a secondary use.

    Low-priority improvements:

    • tools/provision/src/provision/playbooks/wazuh: Wazuh-related playbooks This directory could be re-structured, probably the non install/uninstall actions (register, services, set repo) could be separated on a different folder as these actions are not specifically related to an installation type.
    • tools/provision/tests: Unit Tests suite This suite uses Python's default test runner UnitTests utilities and pytest as the runner, it could be improved by only using pytest and its utilities, which is recommended. Related issues:
      • https://github.com/wazuh/wazuh-qa/issues/4993
    Jenkins

    Low-priority improvements:

    • tools/jenkins/tests: tool's Unit Tests directory Must implement more Unit Tests for this tool
    • tools/jenkins/src/job-builder: Jenkins pipelines as a code Add a README.md detailing this folder, it could be implemented here or in the main Jenkins src directory (tools/jenkins/src), in the second case, it should also includes a summary about bootstrap.

Deliverables

4.10.0 Alpha 2

QA

  • https://github.com/wazuh/wazuh-qa/issues/5556
    • Create repository
    • Create branches: 4.10.0, 4.10.1, 5.0.0
    • Migrate JobFlow (ex Workflow engine) tool
    • Migrate Provision module
    • Migrate release procedures
    • Migrate new Jenkins's code
    • Migrate Tests module and DTT workflows
    • Migrate code from qa-integration-framework
    • Migrate code from qa-system-framework
    • Migrate code from wazuh-tools
    • Migrate code from wazuh-qa:
      • Environments provisioning module
      • System tests - All
      • End to End tests - Only VD
    • Migrate code from wazuh-jenkins:
      • Test_upgrade and Test_upgrade_tier
      • Procedure_ecs_task and Procedure_ecs_task_tier
      • Procedure_gh_project_syncup
      • Test_stress
      • Test_stress_gh
      • Test_stress_tier
      • Test_stress_Vagrant
    • Migrate release issue generation template
  • https://github.com/wazuh/wazuh-jenkins/issues/6811
  • https://github.com/wazuh/wazuh-jenkins/issues/6770
  • https://github.com/wazuh/wazuh-qa-automation/issues/56
  • https://github.com/wazuh/wazuh-qa-automation/issues/26 (Maintenance with all of 4.10.0-alpha1)
  • https://github.com/wazuh/internal-devel-requests/issues/1558
  • https://github.com/wazuh/internal-devel-requests/issues/1555
  • https://github.com/wazuh/wazuh-qa-automation/issues/113
  • https://github.com/wazuh/wazuh-qa-automation/issues/111

DevOps

  • https://github.com/wazuh/wazuh-automation/issues/1658
  • https://github.com/wazuh/wazuh-virtual-machines/issues/22
  • https://github.com/wazuh/wazuh-installation-assistant/issues/21
  • https://github.com/wazuh/wazuh-automation/issues/1766
  • https://github.com/wazuh/wazuh-automation/issues/1767
  • https://github.com/wazuh/wazuh-qa/issues/5654

Agent

  • https://github.com/wazuh/wazuh-agent-packages/issues/55
4.10.0 - Alpha 3

QA

  • https://github.com/wazuh/wazuh-qa-automation/issues/84
  • https://github.com/wazuh/wazuh-qa-automation/issues/221
  • https://github.com/wazuh/wazuh-qa-automation/issues/226
  • https://github.com/wazuh/wazuh-qa/issues/5556
    • Migrate code from wazuh-jenkins:
      • wazuh_qa_environment
      • Procedure_docker_testing_image_generation
      • Test_e2e_system
    • Review the migrated code
    • Update the README.md
    • Maintenance of branches.
    • Create documentation basic documentation
  • https://github.com/wazuh/wazuh-qa-automation/issues/274

Agent

  • https://github.com/wazuh/wazuh-agent-packages/issues/55
4.10.1 - Alpha 1

QA

  • https://github.com/wazuh/wazuh-qa/issues/5556
    • Migrate code from wazuh-jenkins:
      • workloads
      • Test_integration_endpoints

havidarou avatar Apr 08 '24 09:04 havidarou