How to remove duplicate entries from file after merge two separate sbom file?

[satyendra22]

How to remove duplicate entries from file after merge two separate sbom file? I have generated 2 separate sbom file and both files have few common information and i want to create unique entries in sbom file.

[officerNordberg]

yes, I expected these two entries would be merged. I even tried to remove ?type=jar from my PURLs but merge still left them as 2 entries in my combined bom.

` "type": "library", "bom-ref": "pkg:maven/com.fasterxml.jackson.core/[email protected]", "publisher": "FasterXML", "group": "com.fasterxml.jackson.core", "name": "jackson-databind", "version": "2.9.10.6", "purl": "pkg:maven/com.fasterxml.jackson.core/[email protected]?type=jar",

"type": "library", "group": "com.fasterxml.jackson.core", "name": "jackson-databind", "version": "2.9.10.6", "cpe": "cpe:2.3:a:jackson-databind:jackson-databind:2.9.10.6:::::::*", "purl": "pkg:maven/com.fasterxml.jackson.core/[email protected]", `

[software-testing-professional]

I'm also facing this issue.

My use case is scanning a Helm chart repository, and I want to scan all images defined by the chart.

helm template . \
    | grep 'image:' \
    | sed -r 's/image: (.*)/\1/' \
    | sed -r 's/"//g' \
    | while read -r line; do
        filename=$(echo "${line}" | sed -r 's/.*\/(.*):.*/\1/'); \
        docker run \
          -u root \
          --privileged \
          -v /var/run/docker.sock:/var/run/docker.sock \
          tern \
          --driver fuse \
          report \
            -i "${line}" \
            -f cyclonedxjson \
            -o "/project/${filename}.container.bom.cyclonedx.json"; done

This outputs all images that are referenced in the chart.

If the rendered Helm chart contains more than one image, duplicate components are likely created in the merged CycloneDX file. Duplicates occur in the tools and components section in my case.

My workaround here is to remove the duplicates with jq. Therefore I rewrite the CycloneDX file, and remove all duplicate entries in tools and components.

jq '.'
    | {
        "bomFormat": .bomFormat,
        "specVersion": .specVersion,
        "version": .version,
        "metadata": {
          "tools": [(.metadata.tools | unique[])]
        },
        "components": [(.components | unique[])]
      }' "merged.bom.cyclonedx.json" > "final.bom.cyclonedx.json"

I hope this helps. :-)

[candrews]

This was fixed in cyclonedx-dotnet-library by https://github.com/CycloneDX/cyclonedx-dotnet-library/pull/199 which included in the 5.4.0 release of that project.

This project, cyclonedx-cli, updated to version 6.0.0 of the cyclonedx library with https://github.com/CycloneDX/cyclonedx-cli/commit/34382a43ab2a1ca2b9ce5963b50e18f5cbfe8198#diff-bde0c1a112eb6bb84760539e3dd3fe49d502d6b065bbd2efa005a648f6c236fdR18 which is included with the cyclonedx-cli 0.25.0 release.

tl;dr: this issue is fixed in version 0.25.0 of this project.