Decentralize vulnerability database

[gersongoulart]

First of all, congratulations for the amazing initiative. This project rocks! :-)

Based on my understanding from the docs, the source of vulnerabilities is a manually maintained JSON file in your GitHub repository that was created by looking through release notes and issue trackers for the most common frameworks...

That seems to me like a lot of work both for the Retire.js project contributors, who have to manually hunt the vulnerabilities in release notes, and for project owners who have to submit vulnerability information through pull requests after they had already done the write up in the release notes... (which generates more work for contributors who will also have to validate and merged these pull requests).

I am just wondering if it wouldn't make it easier for everybody (and also help the retire.js project to take off) if the vulnerability information was decentralized, and stored perhaps inside the package.json file of the project (or maybe a new retire.json file also located in the root of the project, in a similar approach to what Bower and Grunt currently do)?

If done this way, project owners could maintain this information themselves (possibly even automating the task by having their bug-track software automatically write the .json file), and projects such as NPM and Bower would be able to easily integrate Retire.js and do something useful such as displaying warnings to the users and optionally prevent installation/update of project versions with vulnerabilities.

Just food for thought...

[kozmic]

Thanks for great feedback. I agree with you that this would be a better solution, it could be even better if npm/package.json could have support to mark previous versions with vulnerability information. Both of these solutions shift all the work over to the maintainers of other projects. Maybe a combination would be a good solution, project maintainers can add a retire.json for their project, then retire.js will look for the retire.json in dependencies, in addition to using the database we already have today?

As of today, I think the way retire.js does it is the right way (or maybe the vulnerabillity "database" could be separated out in a website/project to decentralize it a bit further), because it supports all js-libraries/frameworks without any effort from their authors. But I agree that we should try to provide a solution for maintainers of projects with vulnerabilities to provide this information themself (retire.json).

TLDR; +1, add support for retire.json that node modules and bower modules can provide

[gersongoulart]

Great idea @kozmic! I think I had a bit of an "all-or-nothing" mentality while writing the first comment, but you're right: A combination of both solutions sounds like the best way to go!

So the maintainers of other projects could have their vulnerability annotations inside a retire.json file (or in the package.json file, if the former doesn't exist) — at the same time the Retire.js team maintains the database already in use today, perhaps separated out in a website/project that makes it dead easy for the community to contribute new vulnerability info for projects that do not yet maintain their own...

[eoftedal]

This is a great idea! I like the following solution. package.json can include "retire" : "<some uri to the projects retire.json>" When scanning the dependencies, we can then look for this URI in every dependency's package.json, download it and merge it with what's coming from the central repo. Why? If I have installed module x version 1.2.1, and that version becomes vulnerable, how will I know if I don't upgrade? I would then have the retire.json shipped with 1.2.1, which doesn't mark 1.2.1 as vulnerable. If instead package.json defines a URI with vulnerability-info (typically directly into the github repo like we do for the central repo), then the package maintainer can mark version 1.2.1 as vulnerable there, and the developer is guided to upgrade.

[gersongoulart]

Hey @eoftedal!

So, let me see if I clearly understood how the mechanics of Retire.js would work:

• Retire.js would check the root of a 3rd party project for a retire.json file containing vulnerabilities data;

- if that's not found -

• Retire.js would check the root of the 3rd party project for a package.json file and, if it exists, look for a retire object which value could be either:

  • An object containing all the vulnerability information;

    E.g.:

    "retire": {
      "vulnerabilities" : [ ... ],
      "extractors" : { ... }
    }
    

  • Or a string with the relative path to a valid json file containing the vulnerabilities data;

    E.g.:

    "retire": "dist/vulnerabilities.json"
    

• Once Retire.js gets a hold of the local vulnerability data, it would fetch the vulnerability data from Retire.js’ central repository and merge it, kinda like:

  $.extend(true, {}, localVulnerabilityData || {}, remoteVulnerabilityData);

And from here alert the user (or throw exceptions, depending on severity of vulnerability and user-defined tolerance).

Did I get the idea right? If yes, I really love it.

[eoftedal]

Hi @gersongoulart Here's how I see it:

1. For node modules, Retire.js would look for a "retire" element within package.json. If this element exists, it contains a URI to an actual retire.json for the project. "retire" : "http://raw.github.com/user/project/retire.json"
2. If there is no package.json or package.json does not contain a retire element, it would look for retire-repo.json at the root of the project. This file would be a once again define the module name and a URI pointing to the retire.json for the project { "<module name>" : "<uri>" }
3. If the file is a frontend javascript file (not a node module), we could have a special comment pointing to the repo @retire jquery http://raw.github.com/jquery/retire.json (so @retire <module name> <uri>)

As mentioned in my comment above, the vulnerability information cannot be distributed within the package. Version 1.2.1 of module x, will not contain information about itself being insecure, because the vulnerability was likely not identified when the module was first released. So the retire.json within package 1.2.1 will be outdated with regards to itself. Maybe I'm missing something.