Invalid payload submitted to Node Audit API. Received response code: 400 Bad Request

[rowe42]

In our project the dependency-check fails on the Node Audit analyzer with the error:

Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.

However, this is NOT the same problem as in #2641

Investigating I found out that in fact the payload, which is generated in https://github.com/jeremylong/DependencyCheck/blob/main/core/src/main/java/org/owasp/dependencycheck/analyzer/NodeAuditAnalyzer.java#L184, seems not to be accepted by the NPM REST API.

What I did was

• run the dependency check (in my case throught the CLI like this: dependency-check -s . --disableYarnAudit --log dependencycheck.log
• open the file dependencycheck.log and find the payload in there (close to the line Node Audit Payload) and copy it
• run the call to https://registry.npmjs.org/-/npm/v1/security/audits manually with curl or Postman like this

curl --location --request POST 'https://registry.npmjs.org/-/npm/v1/security/audits' \
--header 'Content-Type: application/json' \
--data-raw 'PLACE-PAYLOAD-HERE'

and I get the same error

{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "Invalid package tree, run  npm install  to rebuild your package-lock.json"
}

When I run npm audit it works though, so I think it's not a problem with my package-lock.json or with NPM.

Here is the payload in our example nodeaudit_payload.txt

So, the problem must be somewhere in NpmPayloadBuilder.java.

Stripping down the payload on a trial-and-error basis, the error persists and only goes away when the entries in section requires resemble the entries in section dependencies - but not sure if that is really the rule.

[magob-sagefr]

Hello, I have the very same behavior, meaning that npm audit works fine on the contrary to dependency checker in the node audit analyzer which fails both on my Jenkins server and locally. No more specific information thant "Invalid payload submitted to node audit analyzer". The used API URL is the default one : https://registry.npmjs.org/-/npm/v1/security/audits Any idea ?

[andrzejcichonroche]

Hello, It happens because package.json contains dependency versions which are 'NaN', for example url to git repos or file path. There is a fix for it, if file path contains "file:" prefix, dependencyCheck plugin skips that dependency for building payload for npm audit API.

[ryandutton]

All my dependencies are semver (no paths) and I'm still getting the error above

[ryandutton]

Can confirm there are no NaN's in the payload I'm sending and all versions follow the semver pattern

[Samuel-Schober-USU]

Seems to be the same than https://github.com/jeremylong/DependencyCheck/issues/3717 , does it?

[magob-sagefr]

Looks like. At least the shared package-lock.json file does not contain any strange URL as far as they all refer to npm registry. To make sure, people who posted on #3717 could have a look at detailed error message when the error occurs.

[jonhamm]

Had similar problem.

Did a binary search by repeatedly doing

• remove or add some packages from/to devDependencies or dependencies
• run npm install
• run

                       dependency-check.sh --format XML --format HTML --format JSON \
                               --out ./dependency-check-output\
                               --disableYarnAudit --disableRetireJS \
                               --nodeAuditSkipDevDependencies \
                               --disableNodeJS \
                               --suppress .dependencycheck-suppress.xml \
                               --exclude '**/.angular/**' \
                               --exclude '**/.nxcache/**' \
                               --exclude '**/node_modules/.cache/**' \
                               -s .

until you find the one package that causes the trouble. In my case it was devDependencies: "webpack-stream": "^6.1.2", Upgrading this to ^7.0.0 solved the problem

[jonhamm]

Further investigation revealed that the problem exists with plain npm audit This will first try to make a bulk request - but if you intercept that and make it fail then npm audit will make a request similar to the one made by dependency-check and it will fail in the same way.

Guess I should report this to npm

[ryandutton]

Further investigation revealed that the problem exists with plain npm audit This will first try to make a bulk request - but if you intercept that and make it fail then npm audit will make a request similar to the one made by dependency-check and it will fail in the same way.

Guess I should report this to npm

Maybe I'm misunderstanding your comment that particular packages are causing dependency check AND npm audit to fail but in my case I see failures in dependency check which I don't see using npm audit with exactly the same packages and versions. I'm using npm v8.

[jonhamm]

@ryandutton : Under normal circumstances you will not see any error from npm audit because this command will first issue a bulk request - different endpoint and different format https://docs.npmjs.com/cli/v8/commands/npm-audit#bulk-advisory-endpoint - and only if that fails it will issue a request to the quick audit endpoint (the one used by dependency-check). But if you intercept the HTTP request to the bulk endpoint and fail that, then you will see the quick audit endpoint failing just like you see in dependency-check So maybe we have two issues here:

• one with dependency-check: try to use the bulk end point first just like npm audit
• one with npm : the quick audit endpoint is failing on legitimate package-lock.json (generated by npm!)

by the way - I only saw this occuring after upgrading by Nx repo to Nx 14.3.5 and Angular 14.0.1 node 16.15.0 and npm 8.5.5

[Janpopan]

There is an other ticket around this checker (and the problem ? ) https://github.com/jeremylong/DependencyCheck/issues/4536

[iompo]

@ryandutton : Under normal circumstances you will not see any error from npm audit because this command will first issue a bulk request - different endpoint and different format https://docs.npmjs.com/cli/v8/commands/npm-audit#bulk-advisory-endpoint - and only if that fails it will issue a request to the quick audit endpoint (the one used by dependency-check). But if you intercept the HTTP request to the bulk endpoint and fail that, then you will see the quick audit endpoint failing just like you see in dependency-check So maybe we have two issues here:

* one with `dependency-check`: try to use the bulk end point first just like `npm audit`

* one with `npm` : the quick audit endpoint is failing on legitimate `package-lock.json` (generated by `npm`!)

by the way - I only saw this occuring after upgrading by Nx repo to Nx 14.3.5 and Angular 14.0.1 node 16.15.0 and npm 8.5.5

I also have the same problem updating from node 14.15.4/npm 6.14.10 to node 16.15.1/npm 8.11.0. That was the only change in my code base (except from re-creating the package-lock).

[v1shva]

Would it be a good idea to skip node audit analyser with flag --disableNodeAudit for now until this is fixed? and use audit-ci

[delenikov]

I had the same problem when I upgraded to Node 18+ and npm 8+. The solution with deleting node_modules and package-lock.json, followed by npm install did not work for me.

I came up with this solution, I added the following line of code in the configuration tag for dependency-check-maven plugin in the parent pom.xml, and now its working again.

<nodeAuditAnalyzerUrl>/-/npm/v1/security/advisories/bulk</nodeAuditAnalyzerUrl>

[Shhad]

@delenikov Can you explain why does it solve issue? I've tried it and it works, but I want to understand why

[delenikov]

@delenikov Can you explain why does it solve issue? I've tried it and it works, but I want to understand why

I think this will help https://docs.npmjs.com/cli/v7/commands/npm-audit?v=true#bulk-advisory-endpoint I made use of the new bulk advisory endpoint instead of the plugins default one.

[paul-dyson]

What seemed to work for me was to reinstate the resolved and integrity values in package-lock.json that npm v8 had lost by running

npx [email protected] install && npm install

After which the previously failing Node Audit analysis completed without error.

[Reamer]

@delenikov Are you sure that a node audit execution will occur? I think a MalformedURLException is thrown with your value. Unfortunately I don't see the error message of the exception.

https://github.com/jeremylong/DependencyCheck/blob/a5b2b28128bcde185d1cc7c5f2ed1bada34ddbe2/core/src/main/java/org/owasp/dependencycheck/data/nodeaudit/NodeAuditSearch.java#L90-L94

I have written a small test just in my IDE for this.