kibana icon indicating copy to clipboard operation
kibana copied to clipboard

[Security Solution] Integrates security solution tests with Kibana QA quality gate

Open MadameSheema opened this issue 7 months ago • 8 comments

Summary

Once this PR is merged, when the Kibana QA quality gate is triggered the following tests are going to be executed:

  • API tests tagged with @serverlessQA
  • Cypress tests tagged with @serverlessQA

Currently there are only a few that have been extensively tested and are passing on MKI environments.

The current and future tests:

  • Cover critical paths of the Security Solution application
  • Test behaviours that if broken SDH or blocker issues might be reported by customers.

We have created a run book that has been shared with all the team members of Security Solution and can be found in our internal docs with the title Security Serverless Kibana QA Quality Gate Failure Run Book

Some of the most important points mentioned in the run book:

Below are the channels/groups to be contacted should a test fail:

  • Detection Engine tests:

    • #security-detection-engine
    • @security-detection-engine-team
  • Entity Analytics tests:

    • #security-entity-analytics
    • @security-entity-analytics-team
  • Explore tests:

    • #threat-hunting-explore
    • @security-threat-hunting-explore
  • Investigations tests:

    • #security-threat-hunting-investigations
    • @threat-hunting-investigations
  • Gen AI tests:

    • #security-gen-ai-sharing
    • @security-generative-ai-team
  • Rule Management tests:

    • #security-detection-rule-management
    • @security-detection-rule-management-team
  • Defend Workflows tests:

    • #security-defend-workflows
    • @defend-workflows
  • Geneve detection rules tests:

    • #sec-eng-productivity
    • @sec-eng-prod & @Dom

If nobody responds within 30 minutes: Post the issue in the #security-unified-app channel with the message that team X is not responding and assistance is needed. Tag @sec-eng-prod as well.

Should a failure occur, the Release Manager (RM) will post in the team channell and @ the team responsible. The team, unless in the middle of responding to an SDH Sev1-2, must respond immediately and begin investigation with the RM. Serverless test failures in the Production Quality Release Pipeline are BLOCKING for the entire Kibana Serverless release.

MadameSheema avatar Jul 01 '24 15:07 MadameSheema