kibana icon indicating copy to clipboard operation
kibana copied to clipboard

Failing test: Detection Engine - Exception Workflows Integration Tests - Serverless Env - Essentials Tier.x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/exceptions/workflows/basic_license_essentials_tier/rule_exceptions_execution·ts - Detection Engine - Exception workflows APIs @serverless @serverlessQA @ess rule exceptions execution creating rules with exceptions should be able to execute against an exception list that does include valid case sensitive entries and get back 0 alerts

Open kibanamachine opened this issue 10 months ago • 3 comments

A test failed on a tracked branch

JestAssertionError: expect(received).toEqual(expected) // deep equality

Expected: 10
Received: 0
    at Context.<anonymous> (rule_exceptions_execution.ts:204:45)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16) {
  matcherResult: {
    actual: 0,
    expected: 10,
    message: '\x1B[2mexpect(\x1B[22m\x1B[31mreceived\x1B[39m\x1B[2m).\x1B[22mtoEqual\x1B[2m(\x1B[22m\x1B[32mexpected\x1B[39m\x1B[2m) // deep equality\x1B[22m\n' +
      '\n' +
      'Expected: \x1B[32m10\x1B[39m\n' +
      'Received: \x1B[31m0\x1B[39m',
    name: 'toEqual',
    pass: false
  }
}

First failure: CI Build - main

kibanamachine avatar Apr 26 '24 15:04 kibanamachine

Pinging @elastic/security-detection-engine (Team:Detection Engine)

elasticmachine avatar Apr 26 '24 15:04 elasticmachine

New failure: kibana-on-merge - main

kibanamachine avatar May 10 '24 15:05 kibanamachine

Skipped.

main: 312f578

mistic avatar May 11 '24 00:05 mistic

@yctercero I looked at this briefly in reviewing #189659. I was also unable to reproduce the failure, but since getOpenAlerts only waits for the rule status before doing a refresh and then single search to the alerts index, it's theoretically possible for those alerts to not be available in some extremely rare cases. IMO we should probably do away with that refresh-and-search pattern and swap it for the more robust "search until you get results" pattern that we have elsewhere.

rylnd avatar Aug 05 '24 21:08 rylnd

New failure: kibana-on-merge - main

kibanamachine avatar Aug 29 '24 10:08 kibanamachine

New failure: kibana-on-merge - main

kibanamachine avatar Sep 03 '24 10:09 kibanamachine

New failure: kibana-on-merge - main

kibanamachine avatar Sep 17 '24 11:09 kibanamachine

We continue to not be able to reproduce locally. This seems to flake intermittently. If it's too noisy I'll move to skip.

Thanks @rylnd for again confirming it's not a regression.

yctercero avatar Sep 18 '24 04:09 yctercero

New failure: kibana-on-merge - main

kibanamachine avatar Oct 01 '24 19:10 kibanamachine

New failure: kibana-on-merge - main

kibanamachine avatar Oct 03 '24 17:10 kibanamachine

New failure: kibana-on-merge - main

kibanamachine avatar Oct 09 '24 13:10 kibanamachine

New failure: kibana-on-merge - main

kibanamachine avatar Oct 09 '24 15:10 kibanamachine

New failure: kibana-on-merge - main

kibanamachine avatar Oct 11 '24 12:10 kibanamachine