npm-audit-resolver
npm-audit-resolver copied to clipboard
naugtur
Entries previsouly ignored showing up as new
We seem to be hitting a condition with vulnerabilities that were previously ignored showing up as new in our audit-resolve.json file. It looks like the ID is used as part of the key in the audit-resolve.json file, but lately it looks like these IDs are changing (daily over the last couple of days) which causes our build to keep breaking.
I know this tool is not responsible for the IDs, but I was curious if anyone else was having this issue and/or if there is any plan to refine how the key is generated in the audit-resolve.json file. Maybe the github_advisory_id is more stable?
- Entry from March 8th:
1030656|ember-cli>markdown-it-terminal>markdown-it - Entry needed for same ignored entry for March 9th:
1051572|ember-cli>markdown-it-terminal>markdown-it
Thank you for a such a useful tool!
NPM version: 6.14.12 npm-audit-resolver version: 3.0.0-6
We're seeing this across multiple projects and vulnerabilities, with the IDs (source property) changing every few hours, and came here to raise an issue to see if a different/stable ID could be used.
Here is an example from the last 48 hours with one vulnerability (oldest to newest):
[high] node-fetch: node-fetch is vulnerable ... (1028029)
[high] node-fetch: node-fetch is vulnerable ... (1030641)
[high] node-fetch: node-fetch is vulnerable ... (1033253)
[high] node-fetch: node-fetch is vulnerable ... (1038477)
[high] node-fetch: node-fetch is vulnerable ... (1043701)
Note the IDs are constantly incrementing.
Unless this is an issue with NPM/GitHub that can be resolved (not sure where this would be raised), I agree that a more stable identifier needs to be used as it's rendering this project unusable.
Happy to help on a change if it would be useful and if there's consensus on a solution.
Having looked into this further, we initially thought that this was related to the switch of NPM to GitHub Advisory Database, but dismissed this as it occurred months ago and we've only started seeing this in the last few weeks.
I've then seen that the GitHub Advisory Database repository was created in the last few weeks and has commits every few hours. It may be that they regenerate all their identifiers on any commit/build now or similar.
Either way, it seems that the current source property that's used is no longer stable. The only reference I can see to the GitHub reference in the output of npm audit --json is in the via.url property, so it would need to be parsed from that. Although I'm not familiar with what other via options are possible (if any) with npm audit, so this would need confirming. I can see that github_advisory_id is returned from Yarn's audit functionality but not NPMs.
See the same problem and same proposed solution here: https://github.com/IBM/audit-ci/issues/211
I do see github_advisory_id from NPM when I run npm audit --json so I think it will work if everyone thinks that is the right path forward.
I think we may be seeing something different due to NPM versions and the NPM 7 rewrite of audits.
Example output from NPM 8.1.0:
{
"auditReportVersion": 2,
"vulnerabilities": {
"quill": {
"name": "quill",
"severity": "moderate",
"isDirect": false,
"via": [
{
"source": 1054693,
"name": "quill",
"dependency": "quill",
"title": "Cross-site Scripting in quill",
"url": "https://github.com/advisories/GHSA-4943-9vgg-gr5r",
"severity": "moderate",
"range": "<=1.3.7"
}
],
"effects": [
"react-quill"
],
"range": "<=1.3.7",
"nodes": [
"node_modules/quill"
],
"fixAvailable": {
"name": "react-quill",
"version": "0.0.2",
"isSemVerMajor": true
}
},
"react-quill": {
"name": "react-quill",
"severity": "moderate",
"isDirect": true,
"via": [
"quill"
],
"effects": [],
"range": ">=0.0.3",
"nodes": [
"node_modules/react-quill"
],
"fixAvailable": {
"name": "react-quill",
"version": "0.0.2",
"isSemVerMajor": true
}
}
},
"metadata": {...}
}
npm 7+ is using a totally different endpoint from the registry server.
Looks like the change originates from folding the npm advisory into github advisory. It might be a breaking change for all users. I'm gonna try to reach out to npm folks
and btw audit-resolver v3.0.0-7+ is only going to work with npm v8 and some later versions of 7. there's some versions of npm7 that wont work. should still support v6, but moving from one to another may be a breaking change for your decisions file.
@ruyadorno @darcyclarke help! :scream: Is there a way to convert the old ids to the new ids?
Thanks very much all for the info in this thread. This is hitting us as well.
FWIW, we are also seeing a lot of audit failures when we run with the --production flag that we should not, because the dependencies only come through under devDependencies (transitive deps from our devDependencies, that is). This is solely a bug with NPM, NOT npm-audit-resolver, but this thread was helpful because I believe some of the recent npm audit changes must have caused this.
npm support Ticket ID: 1541272
email thread id GO8PEM-Z34EY
@MylesBorins some folks suggested I could let you know this is happening.
@naugtur we are aware and researching internally to figure out what exactly is going on
Thanks @MylesBorins !
Don't hesitate to loop me in if there's anything I can do to help.
I am not aware of any updates. But I have no insight into npm/github.
Is anybody monitoring if this is still occurring? I switched to other aspects of supply chain security a few months back and npm audit is not part of my daily routine right now.
I'm pondering a rewrite where I skip the dependency on npm/yarn for audit and compile the audit results from github vulnerability database directly, using other identifiers, like the CVE number. It may be prohibitively expensive to do though and would benefit from doing everything from scratch (because a migration path from broken resolve files doesn't make sense anyway)
With the current situation the only valuable part of npm-audit-resolver is the user interface code. I'd throw away everything else if I decided to do the above.
There's a design in my head. Next step: feasibility study. If that goes well, I'd feel comfortable starting this project if I had 2-3 people with a weekly quota of 6-8 hours to work on it. At this time I'm not sure I'd have room to commit to a weekly quota myself.
@naugtur That sounds like an interesting long-term solution.
In the short-term, would you consider switching to using github_advisory_id as the identifier as mentioned earlier in this issue, which would presumably need to go into a major release as a breaking change? This approach has already been adopted by similar projects hitting this issue, e.g. https://github.com/IBM/audit-ci/pull/217. If so, and if it's something you would accept a PR for, it may be something we can assign resources to.
+1 to @adammwood's suggestion.
One point of clarification. I obviously don't know if others have the same opinion, but the only issue for us is that the npm IDs are not stable. We have no problem throwing away all of our audit resolver files and starting over (i.e. things aren't working for us now anyways so we're doing audit failure checks manually), so literally anything that would be unlikely to change, even like a hash of the issue title, would work for us.
For some reason I thought both are not stable. :facepalm: I'll merge a PR for that change right away (or implement myself whenever I get an afternoon to myself)
GitHub advisory ID will be stable. I'm working on getting rid of the mutability of the npm advisory ID.
@creising @adammwood @adevine can any one of you confirm you're no longer having issues? I'd close this one.
@creising release was not necessary. The issue was in data supplied from npm
@naugtur responding on behalf of @adevine to your question from 2022-06-13 -- yes, this seemed to resolve for our project for the past ~6 months.
We are observing behavior this week that may indicate this behavior has regressed. However, I haven't done enough diligence to say this with complete confidence ... I'm still looking into it.
That'd be bad timing, as I was hoping to finally release the v3 and get to work on a brand new way of doing the checking.
Let me take a deeper look at the issues we're seeing -- I'll respond here by 5pm MST with my findings. Hopefully it's not a regression in the npm data source.
Here's a sample config:
{
"decisions": {
"1054663|firebase>@firebase/analytics>@firebase/component>@firebase/util": {
"decision": "ignore",
"madeAt": 1646842490288,
"expiresAt": 1649434452188
},
"1054172|firebase>@firebase/analytics>@firebase/component>@firebase/util>@firebase/app>@firebase/database>@firebase/firestore>node-fetch": {
"decision": "ignore",
"madeAt": 1646842496080,
"expiresAt": 1649434452188
},
"1054183|firebase-admin>node-forge": {
"decision": "ignore",
"madeAt": 1646842497494,
"expiresAt": 1649434452188
},
"1054287|swagger2openapi>better-ajv-errors>jsonpointer": {
"decision": "ignore",
"madeAt": 1646842498850,
"expiresAt": 1649434452188
},
"1054685|swagger2openapi>better-ajv-errors>jsonpointer>oas-validator>ajv": {
"decision": "ignore",
"madeAt": 1646842500173,
"expiresAt": 1649434452188
},
"1054109|urijs": {
"decision": "ignore",
"madeAt": 1646842501645,
"expiresAt": 1649434452188
},
"1054113|urijs": {
"decision": "ignore",
"madeAt": 1646842503398,
"expiresAt": 1649434452188
},
"1070480|firebase-admin>dicer": {
"decision": "ignore",
"madeAt": 1658757840161,
"expiresAt": 1661349828635
},
"1075704|@google-cloud/pubsub>google-gax>protobufjs": {
"decision": "ignore",
"madeAt": 1659021126685
},
"1084677|xml-crypto>@xmldom/xmldom": {
"decision": "ignore",
"madeAt": 1665524576309
},
"1085242|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728035974,
"expiresAt": 1674320022857
},
"1085242|firebase-functions>firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728035974,
"expiresAt": 1674320022857
},
"1085242|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728035974,
"expiresAt": 1674320022857
},
"1085243|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728039600,
"expiresAt": 1674320022857
},
"1085243|firebase-functions>firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728039600,
"expiresAt": 1674320022857
},
"1085243|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728039600,
"expiresAt": 1674320022857
},
"1085244|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728041727,
"expiresAt": 1674320022857
},
"1085244|firebase-functions>firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728041727,
"expiresAt": 1674320022857
},
"1085244|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728041727,
"expiresAt": 1674320022857
},
"1085245|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728044554,
"expiresAt": 1674320022857
},
"1085245|firebase-functions>firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728044554,
"expiresAt": 1674320022857
},
"1085245|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671728044554,
"expiresAt": 1674320022857
},
"1085248|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085248|firebase-functions>firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085248|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085250|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085250|firebase-functions>firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085250|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085251|firebase-admin>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085251|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
}
},
"rules": {},
"version": 1
}
The following occurs when checking audit (moment of execution ~2022-12-28 4:45PM MST):
(base) wokkaflokka@wokkaflokka % npx check-audit --omit=dev
>>>> npm audit --json --omit dev
Total of 4 actions to process
--------------------------------------------------
[high] jsonwebtoken: jsonwebtoken has insecure input validation in jwt.verify function (1085261)
twilio>jsonwebtoken
--------------------------------------------------
😱 Unresolved issues found!
--------------------------------------------------
This isn't an exhaustive analysis but it sure feels like these ID's are once again unstable.
I doubt they are unstable given my experience that every time I check at the source (GitHub Advisory of reviewed vulnerabilities) there is in fact a new vulnerability.
There are four different CVEs reported against jsonwebtoken <= 8.5.1:
https://github.com/advisories?query=type%3Areviewed+jsonwebtoken
Yes, but my N coworkers keep having to ignore day after today (even today, even right now it's complaining despite a commit around 11am CST adding an ignore directive), and several of them indicated they ignored for 30 days (7 days ago). A 30 day ignore in the past week shouldn't notify until late January.
In any case:
- Greatly appreciate your work and this repo.
- I'm on vacation and only touching base for this short window of time. I could totally be wrong, you likely know significantly more in this domain. I just wanted to follow up as I said I would.
It still smells to me like this is an upstream issue, but I have to defer to you as a domain expert here.
Another way to look at jsonwebtoken is to see that there are 4 separate advisories posted in the source code repo:
https://github.com/auth0/node-jsonwebtoken/security/advisories
All reported in the last week by a maintainer of the code.
FWIW, I have verified the upstream IDs are unstable. Yesterday (2022-12-29) throughout the day out builds passed, and then late yesterday evening started failing due to changes in the jsonwebtoken vuln IDs. That is, we previously had these entries in audit-resolve.json:
"1085251|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1671743398973,
"expiresAt": 1674335375940
},
"1085261|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1672249670936,
"expiresAt": 1674841660819
},
When I ran resolve-audit this morning (because it was being reported that these 2 were failing again) it added these 2 entries:
"1085291|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1672416390477,
"expiresAt": 1675008376923
},
"1085292|twilio>jsonwebtoken": {
"decision": "ignore",
"madeAt": 1672416398816,
"expiresAt": 1675008376923
}
(notice that the "madeAt" in these 2 entries is before the "expiresAt" of the previous ones).
I know jsonwebtoken has had a bunch of new vulns reported recently, but something with them is causing the IDs to be unstable when they were previously ignored.
@adevine I don't understand your comment about madeAt
There are currently four different reviewed advisories for jsonwebtoken and you listed four different indices above.
What is the expected result?
Is there a way to match these indices with the GitHub advisory?
Here are the four advisories. Note that two were updated 18 hours ago. Does the index change every time they are updated?
- https://github.com/advisories/GHSA-27h2-hvpr-p74q (updated 2 days ago)
- https://github.com/advisories/GHSA-8cf7-32gw-wr33 (updated last week)
- https://github.com/advisories/GHSA-qwph-4952-7xr6 (updated 18 hours ago)
- https://github.com/advisories/GHSA-hjrf-2m68-5959 (updated 18 hours ago)