vulnerablecode icon indicating copy to clipboard operation
vulnerablecode copied to clipboard

Import data from OpenSSF Malicious Packages

Open shravankshenoy opened this issue 1 year ago • 2 comments

Fixes #1409

Changes Made

Created OpenSSF importer openssf.py and added files for testing the importer

Results

Ran the importer locally using command below with debug mode on docker-compose exec vulnerablecode ./manage.py import vulnerabilities.importers.openssf.OpenSSFImporter

Importer ran successfully and imported 17182 advisories as can be seen in the image below

image

Other Consideration

  1. Initially I named the file ossf.py but it was tantalizingly close to oss_fuzz.py and could cause confusion, hence changed the name to openssf.py. Let me know if any better name is possible
  2. Out of the 4 ecosystems in the repository (crates.io, npm, pypi, rubygems), univers supports 3 (npm, pypi, rubygems), but I am not sure if it supports crates.io
  3. For the time being, I have added the 4 ecosystems in the list, but if required I can create a separate classmethod called supported_ecosystems and add all the ecosystems there as a list/dictionary, similar to how it is done in Vulntotal. Do let me know what is best approach
@classmethod
    def supported_ecosystem(cls):
        return {           
            "pypi": "PIP",
            "gem": "RUBYGEMS",
            "npm": "NPM",           
        }
  1. Have add a try-except block in for loop. Let me know if that has to be removed.

shravankshenoy avatar Feb 05 '24 09:02 shravankshenoy

I have made the changes that had been requested. Let me know if anything else needs to be done.

shravankshenoy avatar Feb 08 '24 04:02 shravankshenoy

While working on PRs #1405 and #1417 I realized that in this PR I have not followed the purl-spec properly. For example crates.io should be mapped to cargo and rubygems should be mapped to gems. I will work on making these changes.

shravankshenoy avatar Feb 12 '24 10:02 shravankshenoy

@shravankshenoy Thanks! PR can not be merged in this state, I tried to run the importer and it's throwing errors. Please run the importer and improver once.

TG1999 avatar Jul 22 '24 15:07 TG1999