CXX-3309 Automate SBOM generation and Endor Labs scanning

[jasonhills-mongodb]

Objective

• Obviate manual SBOM component management via automated generation of a CycloneDX SBOM that includes all required and optional runtime components.
• Add Endor Labs SCA scanning of PRs (non-blocking)

Changes

• Add a GitHub Action workflow (.github/workflows/endor_scan_and_generate_sbom.yml) triggered by edits to cmake files that:
  • (push) performs an Endor Labs SCA scan and exports an SBOM for master or releases/v* branches, enriches SBOM with metadata, opens a PR for updated SBOM
  • (pull_request) performs an Endor Labs SCA scan for PRs (non-blocking) and, if potential vulnerabilities were found, add a comment to PR
  • (workflow_dispatch) workflow can also be triggered manually
• Remove the etc/purls.txt file
  • update all references to it in scripts and documentation
  • existing scripts and processes using Silkbomb for the augmented SBOM are unchanged, except that the sbom.json file is used as input instead of etc/purls.txt
• Change the location of the SBOM file from etc/cyclonedx.sbom.json to sbom.json for consistency with other MDB repos
• Add etc/sbom folder with:
  • Python scripts for SBOM generation using Endor Labs scan results and pre-defined enrichment data
    • Dependency group added to pyproject.toml
  • SBOM enrichment data in etc/sbom/metadata.cdx.json
• Updated SBOM-related documentation in etc/releasing.md

Testing

The workflow was thoroughly tested on a fork, including scenarios with missing or malformed files.

Permissions

The workflow requires the repository configuration to allow it to request write access and open PRs.

Miscellaneous

• Once this is merged to master, it may be cherry picked to the relevant release branches.
• Once approved, this will be ported to the C driver repo.
• The new SBOM format is in alignment with the NITA Minimum Elements for Software Bill of Materials and OWASP Software Component Verification Standard (SCVS). Both publications emphasize that all components used in creation of software are to be documented in an SBOM, including ecosystem-based dependencies, as a best practice.