wazuh-qa icon indicating copy to clipboard operation
wazuh-qa copied to clipboard

Published 20 hours ago • verified publisher icon wazuh

Failures in Vulnerability Detection E2E Tests Beta 6

Open Rebits opened this issue 1 year ago • 1 comments

Description

This issue tracks all known issues responsible for failures in Vulnerability Detection tests. The specified report is utilized to validate the Additional Vulnerability Detection End-to-End tests.

Initial Scans

  • test_first_syscollector_scan[vd_disabled_when_agents_registration]
    • Database errors: https://github.com/wazuh/wazuh/issues/22847
    • centOS arm64 agent did not trigger any vulnerability in E2E VD tests. After removing and registering the agent again, the agent's vulnerabilities seem to appear. This was not possible to reproduce. Following discussions with @davidjiglesias, further investigation is planned during the next 4.8.0 stage.
  • test_consistency_initial_scans
    • Different vulnerabilities between vd_enabled_when_agents_registration and vd_disabled_when_agents_registration. Related to the vd_disabled_when_agents_registration error. Following discussions with @davidjiglesias, further investigation is planned during the next 4.8.0 stage.

Installation of a vulnerable package when agents are down

  • test_install_vulnerable_package_when_agent_down[install_package]
    • Missing vulnerabilities: https://github.com/wazuh/intelligence-platform/issues/1467
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability: https://github.com/wazuh/wazuh/issues/23078
      • Missing CVE-2023-4822 in test package metadata: https://github.com/wazuh/wazuh-qa/issues/5320

Installation of a vulnerable package after agents's manager change

  • test_change_agent_manager[install_package]
    • Missing vulnerabilities: https://github.com/wazuh/wazuh/issues/23192
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
      • Missing CVE-2023-4822 in test package metadata: https://github.com/wazuh/wazuh-qa/issues/5320
      • Missing expected alerts:
        • https://github.com/wazuh/wazuh/issues/23192
        • https://github.com/wazuh/wazuh-qa/issues/5321

Basic Vulnerability Detection cases

  • test_vulnerability_detector_scans_cases[install_package]
    • Missing vulnerabilities: https://github.com/wazuh/wazuh/issues/23192
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
      • Missing CVE-2023-4822 in test package metadata: https://github.com/wazuh/wazuh-qa/issues/5320
    • Missing expected alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
  • test_vulnerability_detector_scans_cases[remove_package]
    • Missing expected mitigated alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
  • test_vulnerability_detector_scans_cases[upgrade_package_maintain_vulnerability]
    • https://github.com/wazuh/wazuh/issues/22867
    • Missing vulnerabilities: https://github.com/wazuh/wazuh/issues/23192
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
      • Missing CVE-2023-4822 in test package metadata: https://github.com/wazuh/wazuh-qa/issues/5320
    • Missing expected alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
    • Missing expected mitigated alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
      • https://github.com/wazuh/wazuh-qa/issues/5312
    • Failed setup operations. Due to previous failed checks
  • test_vulnerability_detector_scans_cases[upgrade_package_add_vulnerability]
    • https://github.com/wazuh/wazuh/issues/22867
    • Missing vulnerabilities
      • https://github.com/wazuh/intelligence-platform/issues/1467
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
      • Missing CVE-2023-4822 in test package metadata: https://github.com/wazuh/wazuh-qa/issues/5320
    • Missing expected alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
    • Missing expected mitigated alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
      • https://github.com/wazuh/wazuh-qa/issues/5312
  • test_vulnerability_detector_scans_cases[upgrade_package_maintain_add_vulnerability]
    • https://github.com/wazuh/wazuh/issues/22867
    • https://github.com/wazuh/wazuh-qa/issues/5333
    • Missing vulnerabilities
      • https://github.com/wazuh/intelligence-platform/issues/1467
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
      • Missing CVE-2023-4822 in test package metadata: https://github.com/wazuh/wazuh-qa/issues/5320
    • Missing expected alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
    • Missing expected mitigated alerts:
      • https://github.com/wazuh/wazuh/issues/23192
      • https://github.com/wazuh/wazuh-qa/issues/5321
      • https://github.com/wazuh/wazuh-qa/issues/5312
  • test_vulnerability_detector_scans_cases[upgrade_package_remove_vulnerability]
    • https://github.com/wazuh/wazuh/issues/22867
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
    • Missing expected mitigated alerts:
      • https://github.com/wazuh/wazuh-qa/issues/5321
      • https://github.com/wazuh/wazuh-qa/issues/5312
  • test_vulnerability_detector_scans_cases[upgrade_package_nonvulnerable_to_nonvulnerable]
    • https://github.com/wazuh/wazuh/issues/22867
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
  • test_vulnerability_detector_scans_cases[upgrade_package_nonvulnerable_to_vulnerable]
    • https://github.com/wazuh/wazuh/issues/22867
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078
    • Missing expected alerts:
      • https://github.com/wazuh/wazuh-qa/issues/5321
      • https://github.com/wazuh/wazuh-qa/issues/5312
  • test_vulnerability_detector_scans_cases[install_package_non_vulnerable]
    • Unexpected vulnerabilities:
      • Unexpected CVE-2023-34111 vulnerability. https://github.com/wazuh/wazuh/issues/23078

Summary

Product

  • https://github.com/wazuh/intelligence-platform/issues/1467
  • https://github.com/wazuh/wazuh/issues/22847
  • https://github.com/wazuh/wazuh/issues/23078
  • https://github.com/wazuh/wazuh/issues/22867

Tests

  • https://github.com/wazuh/wazuh-qa/issues/5320
  • https://github.com/wazuh/wazuh-qa/issues/5333
  • https://github.com/wazuh/wazuh-qa/issues/5321
  • https://github.com/wazuh/wazuh-qa/issues/5312

Automation

  • https://github.com/wazuh/wazuh-jenkins/issues/6445

Rebits avatar Apr 30 '24 12:04 Rebits

LGTM

MARCOSD4 avatar May 07 '24 11:05 MARCOSD4