Release automation design - Tier 1

[damarisg]

Description

In the releases process there are tasks that are repeated with some changes without complexity.

Therefore, we have decided to automate the process as much as possible. To begin with, you need to research the process in detail.

Once we start this, it will help us reduce execution time and members involved.

Functional requirements

• The QA team will try to identify each task. If it corresponds to another team, it will request that it be automated.

• The QA team will try to integrate the tasks with Jenkins and thus automate the different actions that are currently carried out manually. (Actions detailed in the tasks section).

• The QA team will define a new framework for E2E test development. This will help us to reduce the number of teams involved to run them.

Non-functional requirements

• Document and report issues identified during the process.

Implementation restrictions

• Each script and test must include an analysis and report.

• At the end of each task, it should automatically send a message to the c-release channel so that those responsible can analyze the results and, if it fails, try to solve it or create an issue to investigate.

• The members involved must have the necessary permissions and resources to work without delays.

Plan

Release Protocol: Phase 1

• [ ] Wazuh version: version synchronization
• [ ] Wazuh Packages: Creating Packages
• [ ] Wazuh Resources: Providing the Demo Environment
• [ ] Wazuh Testing: Test Plan

Release Protocol: Phase 2

• [ ] Wazuh test: regression

  Analysis > Return to Phase 1 or Advance to Phase 3

Release Protocol: Phase 3

• [ ] Wazuh version: Branch maintenance with the latest changes.
• [ ] Wazuh Resources: AWS Update
• [ ] Wazuh Release: Define the final tag and final Release Notes.
• [ ] Wazuh Publish: publish and verify packages
• [ ] Wazuh Validation: Sanity Tests
• [ ] Wazuh Sync: Clear branches, tags, and release notes.
• [ ] Wazuh Resources: Re-Generation of demo environment and Jenkins.

---

Tasks

Release Protocol: Phase 1

Wazuh version: version synchronization

Description: Create tag of the new version.

• [ ] Create issue automatically.
  • [ ] Create a pipeline to launch script with specific repository and version. It should send a message to the "c-release" channel with a report when the pipeline finished. If it fail, the team responsible for the failure will research and solve it. It must include:
    • [ ] wazuh-qa: Validate updated changelog, boost, create tag, create and publish release notes as pre-release.
    • [ ] wazuh-jenkins: Change shared libraries, create tags, releases, and change the libraries back to how they were.
    • [ ] wazuh-automation: Create tag, and create and publish pre-release.
    • [ ] wazuh-packages: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh-ansible: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh-docker: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh-puppet: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh-kubernetes: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh-kibana-application: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh interface: Validate updated changelog, create tag, create and publish pre-release.
    • [ ] wazuh-documentation: Validate updated changelog, create tags, create and publish preview versions, upload documents.
    • [ ] wazuh-splunk: Validate updated changelog, create tag, create and publish pre-release.

Wazuh Packages: Creating Packages

• [ ] Request special machines to create special packages.
• [ ] Create an issue that builds packages and launches each pipeline and when it finishes send a message to the c-release channel. Packages to create:
  • [ ] wazuh app packages
  • [ ] Build generic packages
  • [ ] Generate HP-UX FTP container
  • [ ] Build special packages
  • [ ] Build and push pre-release dockerhub images
  • [ ] unattended
  • [ ] Verify packages and checksum

Wazuh Resources: Providing the Demo Environment

• [ ] Create Demo Environment

Wazuh Testing: Test Plan

• [ ] Create test issues automatically

---

Release Protocol: Phase 2

Wazuh test: regression

Analysis > Return to Phase 1 or Advance to Phase 3

• [ ] Test cycle with ETA. It contains:
  • [ ] Footprint, Update, Installation, Selinux, Registry, Service, Indexer, System, Demoo Integration , E2E tests.
  • [ ] Revisions
  • [ ] Check if we need to repeat process or advance.

Currently, testing is manual but it could be automated tests.

---

Release Protocol: Phase 3

If everything is fine and It is approved.

Wazuh version: Branch maintenance with the latest changes.

• [ ] Each repository must push changes from the current revision to the master branch, and must update the changelog. This includes: |- wazuh-qa |- wazuh-jenkins |- wazuh-automation |- wazuh- packages |- wazuh |- wazuh-ansible |- wazuh-docker |- wazuh-puppet |- wazuh-kubernetes |- wazuh-splunk |- wazuh-dashboard-plugins |- wazuh-documentation

I suggest creating a pipeline to launch a script with a specific repository and version. And this should send a message to the "c-release" channel with a report when the process finishes.

Wazuh Resources: AWS Update

• [ ] Regenerate resources from the current revision to the master branch.

Wazuh Release: Define the final tag and final Release Notes.

• [ ] Generate final tag and publish the darft of releases. It includes: |- wazuh-qa |- wazuh-jenkins: update shared, restore shared libraries. |- wazuh-automation |- wazuh- packages |- wazuh |- wazuh-ansible |- wazuh-docker |- wazuh-puppet |- wazuh-kubernetes |- wazuh-dashboard-plugins |- wazuh-documentation

• [ ] Publish Release |- Send AMI to analysis |- Launch publish release without unattended |- Launch unattended |- Build and release debug packages |- Build and push docker hub images |- Publish AMI |- Invalidate cache in lives --> AWS Cloud Cache

• [ ] Check Lives packages |- check production with files present |- deploy wazuh installation assistant and check version. |- AMD64 live installation test. |- WPK version checks |- WPK upgrade |- Ami published |- Puppet forge module published with latest version. |- Docker images published.

Wazuh Validation: Sanity Tests

• [ ] Execute Sanity automation and generate a report.

Wazuh Sync: Clear branches, tags, and release notes.

• [ ] Each a repository should clean and update branches, tags, and release notes. It includes:
  • [ ] wazuh-qa - wazuh-packages, wazuh, ansible, puppet, kubernetes, dashboard-plugin, documentation |- Merges desde rama actual a master |- Publish as latest |- Delete all pre-releases |- Remove all tags not el definitivo |- delete branch y no tag |- Check tag y releases notes.
  • [ ] wazuh-docker |- Merges desde actual a master |- Publish as latest |- Delete all pre-releases |- Remove all tags not el definitivo |- delete only the branch |- Update Docker hub readme compatibility matrix |- Delete stages |- Check tag y releases notes.
  • [ ] wazuh-automation |- Publish as latest |- Delete all pre-releases |- Remove all tags not el definitivo |- delete only the branch |- publush demo.com with new version |- Check tag y releases notes.
  • [ ] wazuh-jenkins |- Merges desde rama actual a master |- Publish as latest |- Delete all pre-releases |- Remove all tags not el definitivo |- delete only the branch |- Update Jenkins configuration with new version. |- Build release containers. |- Update test_nightly parameter. |- Check tag y releases notes.

Wazuh Resources: Regeneration of demo environment and Jenkins.

• [ ] Regenerate resources. It includesd: |- From current version to master. |- Demo environment |- Build Jenkins images

Wazuh Publication

• [ ] Publish docs (https://github.com/wazuh/wazuh-documentation/issues/6701)
• [ ] Announce Release to community.

Note:

Case 1:

When working on each phase, identify what needs to be cleaned and what issues created are already involved. Example:

• Issue: The procedure check release must improve the packages check in S3 and EFS
• Issue: The publish release must improve the package handling in EFS

Case 2:

The maintenance is a task that must be do incrementally in each phase. For example:

• Management of maintenance of AWS S3 and EFS packages.

[damarisg]

• [x] Collect general information
• [x] Identify the general process
• [x] Planning
  • [x] Create an initial draft
  • [x] Create a complete draft
  • [x] Identify the plan
  • [x] Identify tasks

[damarisg]

QA - Release Protocol: Phase 1

Wazuh version: version synchronization

Feature: Version Sync

   Scenario Outline: Report Version Sync
     Given a version of Wazuh
     And a Wazuh repository <repository_name>
     And a Wazuh branch
     And a Wazuh team <team_name>
     When I create an issue for <repository_name>
     And I run the <repository_name> script
     Then I see the <repository_name> script finish successfully
     And I see a message in the Slack channel with a report for each <team_name>

    Examples:
      | team_name           | repository_name          |
      | @wazuh/qa           | wazuh-qa                 |
      | @wazuh/qa           | wazuh-jenkins            |
      | @wazuh/cicd         | wazuh-automation         |
      | @wazuh/cicd         | wazuh-ansible            |
      | @wazuh/cicd         | wazuh-docker             |
      | @wazuh/cicd         |wazuh-puppet              |
      | @wazuh/cicd         | wazuh-kubernetes         |
      | @wazuh/frontend     | wazuh-dashboard-plugins  |
      | @wazuh/framework    | qa-integration-framework |
      | @wazuh/documentation| wazuh-documentation      |
      | @wazuh/core         | wazuh                    |
      | @wazuh/core         | wazuh-packages           |

[damarisg]

QA - Release Protocol: Phase 1

Wazuh version: version synchronization

Support a new stage for a specific version in the wazuh-qa repository.

Feature: Version Sync

   Scenario: Keep the change log up to date
     Given a version of Wazuh
     And a review of Wazuh
     And a Wazuh repository
     And a Wazuh branch
     When I go to the change log file
     Then the changelog file has with every version change of wazuh.

   Scenario: Review and version update
     Given a version of Wazuh
     And a review of Wazuh.
     And a Wazuh repository
     And a Wazuh branch
     When I go to the version file
     And I change the wazuh version
     And I change the wazuh review
     Then the version file is updated

   Scenario: Creating the version control tag
     Given a Wazuh repository
     And a Wazuh branch
     And a tag name
     When I create a tag name from Wazuh branch
     Then the label is created successfully

   Scenario: Creating release notes from the version control tag
     Given a version of Wazuh
     And a Wazuh repository
     And a Wazuh label
     When I create release notes as pre-release from the Wazuh tag
     Then the release note is created successfully

[damarisg]

QA - Release Protocol: Phase 1

Wazuh version: version synchronization

Support a new stage for a specific version in the wazuh-jenkins repository.

I define cases for the current state. However, I consider that:

• New Jenkins development should contain Changelog.
• New Jenkins development should contain Review and Version. This will help keep libraries up to date without having to revert shared libraries.

Feature: Version Sync

  Scenario: Update the shared library 
    Given a Wazuh version tag
    And a Wazuh repository
    And a Wazuh branch
    When I modify the current version of the shared libraries to the new version
    Then I see the shared libraries with the new label.
     

  Scenario: Create the version tag
   Given a Wazuh repository
   And a Wazuh branch
   And a tag name
   When I create a tag name from Wazuh branch
   Then I see the tag created successfully.
     

  Scenario: Create release notes from the release tag
   Given a version of Wazuh
   And a Wazuh repository
   And a Wazuh tag
   When I create release notes as pre-release from the Wazuh tag
   Then I see the release note created successfully.

  
  Scenario: Roll back shared library with a specific version
   Given a tag version of Wazuh
   And a Wazuh repository
   And a Wazuh branch
   When I modify the new version of the shared libraries to the current version
   Then I see libraries shared with the previous version.

[damarisg]

QA - Release Protocol: Phase 1

Wazuh Packages: Creating Packages

Recommendation:Task to executed sequentially.

Each task is in a new pipeline to release everything as a set, and if there are failures, the pipeline should stop and send a message to c-release reporting the new problem.

However, each subtask will consist of launching and reporting but the main task of building packages, images or verifying the checksum is another task that will be covered by another script or pipeline that needs to be defined.

Feature: Buid packages

Scenario Outline: create packages
   Given I modified the policies
   And I have the parameters to start each pipeline
   When I build packages <build name>
   Then I see the packages created
   And I see the checksum created
   And I modify the policies
   And I see a message in the Slack channel with a report 

 Examples:
      | build name                    |
      | wazuh app                     | 
      | generic packages              |
      | Generate HP-UX FTP container  | 
      | special packages              | 
      | pre-release dockerhub images  | 
      | unattended                    | 

[damarisg]

QA - Release Protocol: Phase 1

Wazuh Resources: Providing the Demo Environment

  Scenario: Create a demo environment
    Given I have a wazuh review
    And I have a wazuh version
    When I set up the demo environment
    Then I see the demo environment
    And I see a message in the Slack channel with a report 

[damarisg]

QA - Release Protocol: Phase 1

Wazuh Testing: Test Plan

  Scenario: Create a Test Plan
    Given I have a wazuh review
    And I have a wazuh version
    And I have wazuh team 
    When I execute the script to create issues
    Then I see issues created
    And I see a message in the Slack channel with a each issue assigned for each <team_wazuh> team.

[damarisg]

CONCLUSION - Phase 1:

In this first investigation you can see a general idea of all the tasks that must be carried out.

After a call to sync, we'll add whatever details we decide to get started.

Currently, Phase 1 has been added with general cases that the QA team should start working on.