Verify the content of downloaded node dist

[bbrouwer]

trafficstars

Multiple times I have seen a partial version of node installed in {local-repo}/com/github/eirslett/node/... both on Linux and Windows. Removing this file usually fixes the problem, but it is happening more and more frequently.

What I am experiencing is that a build just hangs forever (I just had one hung for 2 days). The last logs I see are this:

[INFO] Downloading https://nodejs.org/dist/v6.11.2/node-v6.11.2-linux-x64.tar.gz to /redacted/target/local-repo/com/github/eirslett/node/6.11.2/node-6.11.2-linux-x64.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly

This is immediately after a successful build and then cleaning out the local Maven artifact repo.

On linux, what I saw this last time in the local repo is a node-v6.11.2-linux-x64.tar.gz file that is about 2.9 MB, not the 13.1 MB that should be there. Looking at the contents of the file, it all looks correct other than it is truncated at 2.9 MB. I'm looking at the code a bit, but didn't dig too far, but I am not sure what would cause a partial file to be downloaded. Does the CloseableHttpResponse start writing to the .tar.gz file before it has finished downloading the content?

I am asking for:

1. Provide a timeout so that the download of node is aborted if the network hangs
2. Download the file to a temp file in the target directory
3. Verify the contents of the file
4. Rename the temp file once complete

I see node provides a SHASUM256.txt.asc files that could be used to verify the contents.

There is no proxy in the way.

Do you want to request a feature or report a bug? Bug

What is the current behavior? Build hangs

If the current behavior is a bug, please provide the steps to reproduce. Probably the easiest way to reproduce would be to point to an alternate URL of an HTTP server that is configured to simply hang forever.

What is the expected behavior? Abort the build after a timeout and not leave behind unverified node binaries that cause future builds to fail.

Please mention your frontend-maven-plugin and operating system version. frontend-maven-plugin version 1.5 Linux 3.10.0-514.21.2.el7.x86_64 #1 SMP Sun May 28 17:08:21 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

[AndreasHae]

Very much in favor of this.

For some reason the download from nodejs.org is very slow today from within my company network, so we ran into the error "zip END header not found" when running install-node-and-npm. The only explanation I can come up with is that the download timed out and the plugin tried to read the (incomplete) downloaded zip file. We're using Windows machines.

If the file was verified after the download completed, we could at least provide a more detailed error message saying that the file is corrupt.

[eirslett]

Yeah, nodejs have had issues with their CDN lately. I agree there needs to be a better solution. Verifying the shasums would make a lot of sense. Maybe the easiest quick-fix is to fetch the whole payload and store it in memory before writing it to disk? In today's version of the plugin, it takes the server response and streams it to disk continuously.

Does the CloseableHttpResponse start writing to the .tar.gz file before it has finished downloading the content?

Yes exactly, that's what is happening. In hindsight, it was probably a bad design decision.