Integrate with National Vulnerability Database

[micahflee]

Flock will have a database of what versions of software everyone has installed. NIST's National Vulnerability Database (NVD) provides a JSON feed of new vulnerabilities: https://nvd.nist.gov/

We should integrate Flock with the NVD. Maybe the keybase bot can post a daily report of which users are running vulnerable software.

[micahflee]

I'm looking into what it will take to make this work. I think to begin with, Flock Server needs it's own copy of the NVD.

First we'll need to download the the last several years of CVE data from https://nvd.nist.gov/vuln/data-feeds -- there is a different file for each year (e.g. nvdcve-1.1-2018.json.gz for all CVE vulns in 2018), and import all of this data into a database.

On a regular basis (probably once a day), we'll need to check the .meta files for CVE-Modified and CVE-Recent, and if they've changes since the last time we checked, download them and update the database, to make sure it's always up-to-date.

This is just the database of vulnerabilities (CVEs). Each CVE includes a list of "known affected software configurations", referencing the software by its Common Platform Enumeration (CPE). An example CPE for LibreOffice is cpe:2.3:a:libreoffice:libreoffice:6.2.1.1:*:*:*:*:*:*:*.

Here's more info on how CPEs work: https://nvd.nist.gov/products/cpe

So in addition to CVEs, the database will also need to a table of CPEs. That CPE link includes download links for recent versions of the "Official CPE Dictionary".

Like with CVEs, we'll need to start by downloading the dictionary, then on a regular basis (daily) make sure it's up-to-date. The CPE dictionary also has a .meta file to check for changes.

And then there's also the CPE Match Feed. The seems to hold dictionaries that match a CPE with wildcards to a list of exact CPEs, I think? Like this:

{
  "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:*:*:*:*:*:*:*:*",
  "versionEndIncluding" : "3.3.2",
  "cpe_name" : [ {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:-:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.2.99.2:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.2.99.3:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.3.0.1:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.3.0.2:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.3.0.3:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.3.0.4:*:*:*:*:*:*:*"
  }, {
    "cpe23Uri" : "cpe:2.3:a:libreoffice:libreoffice:3.3.1.1:*:*:*:*:*:*:*"
  } ]
}

So, once we have all of the CVE and CPE data in a query-able database, the next step is much simpler. Checking for vulnerable software would be something like this:

• For each user:
• Make a list of software, matching names to versions (from os_version, installed_applications, browser_plugins, safari_extensions, opera_extensions, chrome_extensions, and firefox_addons)
• For each piece of software, query the CPE database to see if we can find a CPE that matches it
• If we have a match, query the CVE database (maybe joining with the CPE match data to handle wildcards) to see if there are any vulnerabilities

At the end, we should be able to have a list of vulnerable software installed for each user.

I think that a SQL database is a better choice for data that's this structured compared to elasticsearch. So we should probably use something like postgres with SQLAlchemy.

[micahflee]

I'm trying to see if there are any python libraries that could simplify this work, but I'm not finding anything obvious. This is a related project: https://github.com/jeremylong/DependencyCheck