vulnerablecode
vulnerablecode copied to clipboard
nexB
API performance issues (vulnerabilities endpoint)
A simple Vulnerability details view can take over 3min to be returned, for example https://public.vulnerablecode.io/api/vulnerabilities/516832
This needs to be optimized to make the API usable.
See also these related issues:
- https://github.com/aboutcode-org/vulnerablecode/issues/1538
- https://github.com/aboutcode-org/dejacode/issues/94#issuecomment-2298445423
While page 1 loads in a couple of seconds: https://public.vulnerablecode.io/api/vulnerabilities
Going to page 2 or 3 get stuck and never render.
This is done now!
PRs for references: https://github.com/aboutcode-org/vulnerablecode/pull/1701 https://github.com/aboutcode-org/vulnerablecode/pull/1631 https://github.com/aboutcode-org/vulnerablecode/pull/1558
To test this
We have a new endpoint deployed and live on https://public.vulnerablecode.io/api/v2/
/api/v2/vulnerabilities- https://public.vulnerablecode.io/api/v2/vulnerabilties
Vulnerabilities endpoint- This endpoint has two filters
- alias - where we pass an alias and get vulnerability IDs associated with that alis
- vulnerability_id - where we pass vulnerability ID and get info for the VCID
Format:
https://public.vulnerablecode.io/api/v2/vulnerabilities
- In results, we now provide "vulnerabilities" map which has key as VCID and value in this format
{
"vulnerability_id": "VCID-111c-u9bh-aaac",
"url": "http://public.vulnerablecode.io/api/v2/vulnerabilities/VCID-111c-u9bh-aaac"
}
- The above "url" points to information about that vulnerability ID
- For a single Vulnerability ID http://public.vulnerablecode.io/api/v2/vulnerabilities/VCID-111c-u9bh-aaac format looks like this
{
"vulnerability_id": "VCID-111c-u9bh-aaac",
"aliases": [
"CVE-2017-1000136"
],
"summary": "Mahara 1.8 before 1.8.6 and 1.9 before 1.9.4 and 1.10 before 1.10.1 and 15.04 before 15.04.0 are vulnerable to old sessions not being invalidated after a password change.",
"severities": [
{
"url": "https://api.first.org/data/v1/epss?cve=CVE-2017-1000136",
"value": "0.00083",
"scoring_system": "epss",
"scoring_elements": "0.36574",
"published_at": "2024-11-01T00:00:00Z"
},
{
"url": "https://api.first.org/data/v1/epss?cve=CVE-2017-1000136",
"value": "0.00083",
"scoring_system": "epss",
"scoring_elements": "0.36853",
"published_at": "2024-11-18T00:00:00Z"
},
{
"url": "https://nvd.nist.gov/vuln/detail/CVE-2017-1000136",
"value": "4.3",
"scoring_system": "cvssv2",
"scoring_elements": "AV:N/AC:M/Au:N/C:N/I:P/A:N"
},
{
"url": "https://nvd.nist.gov/vuln/detail/CVE-2017-1000136",
"value": "6.5",
"scoring_system": "cvssv3",
"scoring_elements": "CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:H/A:N"
}
],
"weaknesses": [
{
"cwe_id": "613",
"name": "Insufficient Session Expiration",
"description": "According to WASC, Insufficient Session Expiration is when a web site permits an attacker to reuse old session credentials or session IDs for authorization."
}
],
"references": [
{
"url": "https://api.first.org/data/v1/epss?cve=CVE-2017-1000136",
"reference_type": "",
"reference_id": ""
},
{
"url": "https://bugs.launchpad.net/mahara/+bug/1363873",
"reference_type": "",
"reference_id": ""
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.10.0:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.10.0%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.10.0:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.10:rc1:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.10%3Arc1%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.10:rc1:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:15.04:rc1:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A15.04%3Arc1%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:15.04:rc1:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:15.04:rc2:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A15.04%3Arc2%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:15.04:rc2:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8.0:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8.0%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8.0:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8.1:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8.1%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8.1:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8.2:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8.2%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8.2:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8.3:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8.3%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8.3:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8.4:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8.4%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8.4:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8.5:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8.5%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8.5:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8:rc1:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8%3Arc1%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8:rc1:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.8:rc2:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.8%3Arc2%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.8:rc2:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.9.0:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.9.0%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.9.0:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.9.1:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.9.1%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.9.1:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.9.2:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.9.2%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.9.2:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.9.3:*:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.9.3%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.9.3:*:*:*:*:*:*:*"
},
{
"url": "[https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe:2.3:a:mahara:mahara:1.9:rc1:*:*:*:*:*:*](https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Amahara%3Amahara%3A1.9%3Arc1%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A%3A%2A)",
"reference_type": "",
"reference_id": "cpe:2.3:a:mahara:mahara:1.9:rc1:*:*:*:*:*:*"
},
{
"url": "https://nvd.nist.gov/vuln/detail/CVE-2017-1000136",
"reference_type": "",
"reference_id": "CVE-2017-1000136"
}
]
}
Additionally we have significantly reduced number of queries to 60% from https://github.com/aboutcode-org/vulnerablecode/commit/7fa45cb0d9dc802a6057edfb003a9f85cfed95fb#diff-d3ca0948dc3b5eb0b1adecaa9da9d7854628b0b6bbcf5f515bed6cab4d894339R474 to https://github.com/aboutcode-org/vulnerablecode/commit/9702c60bb4bac2b98dd988a47948408a16b2cff3#diff-d3ca0948dc3b5eb0b1adecaa9da9d7854628b0b6bbcf5f515bed6cab4d894339R472
Also added indexes for models
