OSV mirror produces invalid / ambiguous affected version ranges

[nscuro]

The OSV mirroring functionality can produce invalid or ambiguous vers ranges. This appears to be at least partially due to data quality issues in the OSV database.

Examples

GHSA-h4w9-6x78-8vrj (ecosystem: Go)

• ranges says: >=1.0.0, <2.1.16
• database_specific says: <= 1.8.7
• mirror-service currently produces: vers:golang/>=1.0.0|<2.1.16|<= 1.8.
  • The last digit of the database_specific range is missing
  • < or <= can only be followed by > or >=, so this range is invalid
  • Could be brought into a valid form again by simplification: vers:golang/>=1.0.0|<2.1.16
    • Simplification requires constraints to be sorted by version, which is ecosystem-dependent
    • It is questionable if vers:golang/>=1.0.0|<2.1.16 would even be correct, as the provided ranges vary drastically in scope

"affected": [
  {
    "package": {
      "ecosystem": "Go",
      "name": "github.com/argoproj/argo-cd"
    },
    "ranges": [
      {
        "type": "ECOSYSTEM",
        "events": [
          {
            "introduced": "1.0.0"
          },
          {
            "fixed": "2.1.16"
          }
        ]
      }
    ],
    "database_specific": {
      "last_known_affected_version_range": "<= 1.8.7"
    }
  }
]

GHSA-g42g-737j-qx6j (ecosystem: Go)

• ranges says: >=0, <1.18.18
• database_specific says: <=1.18.17
• mirror-service currently produces: vers:golang/>=0|<1.18.18|<= 1.18.1
  • The last digit of the database_specific range is missing
  • < or <= can only be followed by > or >=, so this range is invalid
  • Could be brought into a valid form again by simplification: vers:golang/>=0|<=1.18.17
    • Simplification requires constraints to be sorted by version, which is ecosystem-dependent

"affected": [
  {
    "package": {
      "ecosystem": "Go",
      "name": "k8s.io/kubernetes"
    },
    "ranges": [
      {
        "type": "ECOSYSTEM",
        "events": [
          {
            "introduced": "0"
          },
          {
            "fixed": "1.18.18"
          }
        ]
      }
    ],
    "database_specific": {
      "last_known_affected_version_range": "<= 1.18.17"
    }
  }
]

GHSA-f92v-grc2-w2fg (ecosystem: Go)

• ranges says: >=0, <0.18.0
• database_specific says: <= 0.17.2
• mirror-service currently produces: vers:golang/>=0|<0.18.0|<= 0.17.
• The last digit of the database_specific range is missing
• < or <= can only be followed by > or >=, so this range is invalid

"affected": [
  {
    "package": {
      "ecosystem": "Go",
      "name": "github.com/evmos/ethermint"
    },
    "ranges": [
      {
        "type": "ECOSYSTEM",
        "events": [
          {
            "introduced": "0"
          },
          {
            "fixed": "0.18.0"
          }
        ]
      }
    ],
    "database_specific": {
      "last_known_affected_version_range": "<= 0.17.2"
    }
  }
]

GO-2022-0470 (ecosystem: Go)

• ranges says: >=0
• database_specific holds a list of imports and symbols
• mirror-service currently produces: vers:golang/>=0
  • Communicating >=0 without the additional context of the imports is useless and will lead to lots of false positives
  • DT (currently) has no means of evaluating call stacks
• For cases like this, no affected range should be communicated at all

"affected": [
  {
    "package": {
      "name": "github.com/blevesearch/bleve",
      "ecosystem": "Go"
    },
    "ranges": [
      {
        "type": "SEMVER",
        "events": [
          {
            "introduced": "0"
          }
        ]
      }
    ],
    "ecosystem_specific": {
      "imports": [
        {
          "path": "github.com/blevesearch/bleve/http",
          "symbols": [
            "AliasHandler.ServeHTTP",
            "CreateIndexHandler.ServeHTTP",
            "DebugDocumentHandler.ServeHTTP",
            "DeleteIndexHandler.ServeHTTP",
            "DocCountHandler.ServeHTTP",
            "DocDeleteHandler.ServeHTTP",
            "DocGetHandler.ServeHTTP",
            "DocIndexHandler.ServeHTTP",
            "GetIndexHandler.ServeHTTP",
            "ListFieldsHandler.ServeHTTP",
            "SearchHandler.ServeHTTP"
          ]
        }
      ]
    }
  }
]

MAL-2023-995 (ecosystem: npm)

• ranges says: >0
• versions says: 103.99.99
• mirror-service currently produces: vers:npm/>= 0
  • This is incorrect as clearly only a single version is affected
  • OSV claiming a range of >0 is wrong

"affected": [
  {
    "package": {
      "ecosystem": "npm",
      "name": "yandex-yt-yson-bindings"
    },
    "ranges": [
      {
        "type": "SEMVER",
        "events": [
          {
            "introduced": "0"
          }
        ]
      }
    ],
    "versions": [
      "103.99.99"
    ]
  }
]

Conclusions

• For advisories sourced from GitHub (GHSA-*), the database_specific.last_known_affected_version_range field can be necessary to complete a ranges field expressing >=0
  • However, sometimes database_specific.last_known_affected_version_range is specified in addition to an already existing upper bound in ranges, in which case appending it to the vers yields an invalid vers (unless simplified)
• Some advisories will specify a ranges field expressing >=0, but include additional metadata (e.g. imports, call stacks) instead of an upper bound
  • ranges should be ignored in this case
• Some advisories will specify a ranges field expressing >=0, but also explicit versions
  • ranges should be ignored in this case

[nscuro]

I'm trying to address a few issues in #756, but to make this bullet proof we'll need a proper vers implementation takes care of constraint sorting, simplification, and validation for us.