opensearch-build icon indicating copy to clipboard operation
opensearch-build copied to clipboard

Add automated validation during releases

Open bbarani opened this issue 2 years ago • 1 comments

Is your feature request related to a problem? Please describe

We are currently doing manual validation for verifying the artifacts, download links and signatures during release process. This is taking additional time and adding manual overhead during release process.

Describe the solution you'd like

Automate the below validation and verification process during OpenSearch releases,

  • [x] #2712
  • [x] #2883
  • [x] #2759
  • [x] #2858
  • [ ] #2859
  • [x] #2928
  • https://github.com/opensearch-project/opensearch-build/issues/2547

Acceptance criteria:

The automated validation workflow should:

  • [x] Automatically validate download URL's of OpenSearch artifacts
  • [x] Download the OpenSearch distribution artifacts on DockerHub and ECR automatically
  • [x] Validate the checksum of DockerHub and ECR images
  • [x] Start and test the cluster after deploying the artifacts (TAR, Docker, RPM, Windows) for both X64 and ARM64
  • [ ] Validate the native plugin installation using TAR and Windows artifacts.
  • [ ] Signature validation.
  • [x] Validate the artifact(s) signatures for all the supported artifacts including TAR, Docker, RPM, Windows
  • [ ] Verify the presence of released artifacts in maven nexus releases repo.

Describe alternatives you've considered

No response

Additional context

Need a cypress test to test the https://opensearch.org/ website clickstream links.

bbarani avatar Aug 29 '22 21:08 bbarani

Proposal:

I believe it is better to have a validation framework (something similar to https://github.com/opensearch-project/opensearch-build/blob/main/src/sign_workflow/signer.py) with abstract methods like download, install, start_cluster, run_apis, etc. This will allow us to have a uniform execution mechanism across different platforms like docker, rpm, tarballs, windows, macos, etc. This framework then can be wrapped by a jenkins build library which can be called in CI workflows to run validation after each release. cc: @Divyaasm

gaiksaya avatar Oct 12 '22 00:10 gaiksaya