Installing npm fails in a new 14.6.1 project

[OlliTietavainenVaadin]

When building a fresh project with Vaadin 14.6.1 the npm or pnpm command is missing in the command used to install npm, only the arguments are used by the command builder Falling back to Vaadin 14.5.2 to do the initial build installs npm

Stacktrace:

[INFO] --- vaadin-maven-plugin:14.6.1:prepare-frontend (default) @ pws-rgd ---
[WARNING] Error checking if npm is new enough
com.vaadin.flow.server.frontend.FrontendUtils$UnknownVersionException: Unable to detect version of npm. Using command --no-update-notifier --no-audit --scripts-prepend-node-path=true --version
at com.vaadin.flow.server.frontend.FrontendUtils.getVersion(FrontendUtils.java:830)
at com.vaadin.flow.server.frontend.FrontendTools.validateNodeAndNpmVersion(FrontendTools.java:397)
Caused by: java.io.IOException: Cannot run program "--no-update-notifier": CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at com.vaadin.flow.server.frontend.FrontendUtils.getVersion(FrontendUtils.java:820)

According to @caalador "there shouldn't be any changes that would affect this between 2.5 and all the way to 2.7 "

[OlliTietavainenVaadin]

Potential cause:

protected static FrontendVersion getVersion(String tool, List<String> versionCommand) throws UnknownVersionException {
  try {
    Process process = FrontendUtils.createProcessBuilder(versionCommand)
      .start();
    int exitCode = process.waitFor();
    if (exitCode != 0) {
      throw new UnknownVersionException(tool,
        "Using command " + String.join(" ", versionCommand));
    }
    String output = streamToString(process.getInputStream());
    return new FrontendVersion(parseVersionString(output));
  } catch (InterruptedException | IOException e) {
    throw new UnknownVersionException(tool,
      "Using command " + String.join(" ", versionCommand), e);
  }
}

getVersion doesn't use the tool parameter to create the command to execute.

[caalador]

The versionCommand list should contain also the path for the tool as found using getNpmExecutable and the tool String is only for the exception.

[caalador]

Can not get to replicate. Removing all node and npm installations from the computer I get

[INFO] --- vaadin-maven-plugin:14.6.1:prepare-frontend (default) @ project-base ---
[INFO] Couldn't find node.exe. Installing Node and NPM to C:\Users\mikae\.vaadin.
[INFO] Installing node version v14.15.4
[INFO] Downloading https://nodejs.org/dist/v14.15.4/node-v14.15.4-win-x64.zip to C:\Users\mikae\.vaadin\node-v14.15.4-win-x64.zip
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] Unpacking C:\Users\mikae\.vaadin\node-v14.15.4-win-x64.zip into C:\Users\mikae\.vaadin\node\tmp
[INFO] Copying node binary from C:\Users\mikae\.vaadin\node\tmp\node-v14.15.4-win-x64\node.exe to C:\Users\mikae\.vaadin\node\node.exe
[INFO] Extracting NPM
[INFO] Local node installation successful.

Is there any other variables like project, OS or perhaps m2 cache that might be an issue here?

[R-Bosscher]

We use Windows Server 2016 with npm installed in 'D:\Program Files\nodejs If we debug the prepare-frontend we do not see that node.exe is not found, but the function getNpmExecutable in FronendTools executes the statement " List<String> returnCommand = new ArrayList<>( getNpmCliToolExecutable(NpmCliTool.NPM)); ". but returnCommand is empty. There seems to be a mismatch between detecting if a npm,exe is present and locating the command to execute?

[R-Bosscher]

The call to getNodeExecutable() returns "D:\Program Files\nodejs\node.exe".

[pleku]

One related change between the versions is https://github.com/vaadin/flow/commit/3d6b009062059f740a0b52ef26819ca5f0f6e3f2 but it is not at glance clear what / how this could change this. Needs to be investigated deeper.

If we debug the prepare-frontend we do not see that node.exe is not found, but the function getNpmExecutable in FronendTools executes the statement " List returnCommand = new ArrayList<>( getNpmCliToolExecutable(NpmCliTool.NPM)); ". but returnCommand is empty.

The call to getNodeExecutable() returns "D:\Program Files\nodejs\node.exe".

The code for detecting npm is different from detecting the node itself indeed... (in https://github.com/vaadin/flow/blob/2.6/flow-server/src/main/java/com/vaadin/flow/server/frontend/FrontendTools.java#L648) What happens if you try in the windows 2016 server to use where npm.cmd ? The next part will try to run {user-home}/.vaadin/node/node_modules/npm/bin/npm-cli.js for, which is pointless here I think.

I don't know if a simple fix would be to just catch the discovered node location and simply reuse it, but I'm not familiar enough with the current codebase. Maybe @caalador can point out why it cannot be done ? Is pnpm installed to another location than the global npm ? But Node always comes with npm so this should not be a problem for it ?

[pleku]

@OlliTietavainenVaadin @R-Bosscher is this still an issue or not ? We would need code and steps to reproduce it be able to do anything for this.

[R-Bosscher]

It is still a problem... C:\Users\rbo138>where npm.cmd D:\Program Files\nodejs\npm.cmd

[R-Bosscher]

Steps to reproduce:

• Install npm in a different location (e.g. d:\program files)
• Remove previous npm installation from user Vaadin directory
• Start build using Vaadin flow 14.6.1

[R-Bosscher]

When a build is done for Vaadin flow 14.5.x npm is installed in local user Vaadin directory and then a build for vaadin 14.6.1 will succeed.

[pleku]

I'm not still able to reproduce with Windows 10 and I have

 Directory of C:\Users\Pekka\.vaadin

17/06/2021  13.42    <DIR>          .
17/06/2021  13.42    <DIR>          ..
02/06/2020  14.01                83 proKey
               1 File(s)             83 bytes
               2 Dir(s)  447 652 687 872 bytes free
C:\Users\Pekka\.vaadin>where npm.cmd
C:\Program Files\nodejs\npm.cmd
C:\Users\Pekka\.vaadin>nvm list

  * 14.17.0 (Currently using 64-bit executable)

And when starting/building a project, I just get

[INFO]
[INFO] --- vaadin-maven-plugin:14.6.3:prepare-frontend (default) @ project-base ---
[INFO]

I don't know if the issue is specific to Windows Server 2016 (instead of Windows 10), if so, I would appreciate instructions for setting it up locally.

It is still a problem... C:\Users\rbo138>where npm.cmd D:\Program Files\nodejs\npm.cmd

I admit that I'm just guessing, but what if you install the node to the same drive as where the application is ? Another thing - what if you install the node+npm with nvm https://github.com/coreybutler/nvm-windows/release ?

[R-Bosscher]

Tried to install nodejs on c:\program files\ and using nvm but issue remains.

C:\Users\rbo138>where node C:\Program Files\nodejs\node.exe

C:\Users\rbo138>nvm -list 'nvm' is not recognized as an internal or external command, operable program or batch file.

C:\Users\rbo138>where npm C:\Program Files\nodejs\npm C:\Program Files\nodejs\npm.cmd

C:\Users\rbo138>npm --version npm info it worked if it ends with ok npm info using [email protected] npm info using [email protected] npm timing npm Completed in 68ms npm info ok 6.14.8

Run Build results in [WARNING] javax.servlet.ServletException: com.vaadin.flow.server.ExecutionFailedException: Command 'npm install' failed to finish at com.vaadin.flow.server.startup.DevModeInitializer.initDevModeHandler(DevModeInitializer.java:339) at com.vaadin.flow.server.startup.DevModeInitializer.process(DevModeInitializer.java:213) at com.vaadin.flow.server.startup.ClassLoaderAwareServletContainerInitializer.lambda$onStartup$2(ClassLoaderAwareServletContainerInitializer.java:81)

Deinstall npm, after install mvm: C:\Users\rbo138>where npm.cmd INFO: Could not find files for the given pattern(s).

After reinstall node.js: C:\Users\rbo138>where npm.cmd C:\Program Files\nodejs\npm.cmd

C:\Users\rbo138>nvm list No installations recognized.

Reinstall nvm gives waarning that nodejs version is already installed: nvm list

• 14.15.0 (Currently using 64-bit executable)

Running build results in: [INFO] --- vaadin-maven-plugin:14.6.3:prepare-frontend (default) @ pws-rgd --- [WARNING] Error checking if npm is new enough com.vaadin.flow.server.frontend.FrontendUtils$UnknownVersionException: Unable to detect version of npm. Using command --no-update-notifier --no-audit --scripts-prepend-node-path=true --version at com.vaadin.flow.server.frontend.FrontendUtils.getVersion(FrontendUtils.java:830) at com.vaadin.flow.server.frontend.FrontendTools.validateNodeAndNpmVersion(FrontendTools.java:397) at com.vaadin.flow.plugin.maven.PrepareFrontendMojo.execute(PrepareFrontendMojo.java:153)

We can get a working build when we deinstall nodejs, then npm is installed to user .Vaadin directory

So it seems related to using Windows Server 2016...

[pleku]

So it seems related to using Windows Server 2016...

It seems so. For us to be able to work on this, it would require that we can debug a Flow build on Windows Server 2016. While we don't list it explicitly as a supported platform, the platform states that Windows is an supported operating system for development. The supported build systems should probably be listed explicitly, ping @manolo.

For now I'll mark this as an investigation which would start with figuring out how we can get access to the windows server. Up for @mstahv to determine the priority for this as it will take time. In the mean time, probably the expertise team EOD hours can be used for trying to debug the issue.

[mstahv]

Windows Server EOL is just half year away... https://endoflife.software/operating-systems/windows/windows-server

Maybe that could be looked briefly using a remote vm via Azure, instead of spending time to install WS 2016 locally 🤔 https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2016

[manolo]

@pleku windows versions are already mentioned in the release notes https://github.com/vaadin/platform/releases/20.0.0 but it's in the section of the browsers. What else should be done here ?

[mshabarov]

This requires a further investigation and the reproducible case.

[wlfbck]

This is going to be a long one, but i hope this may help some.

The bug is unfortunately not related to Windows Server 2016, but to a bug in Windows itself it seems. If your nvm/node/npm are getting added to the PATH via environment variables themselves. And it seems that sometimes, Windows simply does not resolve those, compare left and right side here:

And it seems to be also somewhat random, as you can see it was able to resolve them on the right side but not the left. My guess is that this is somehow related to them being at the end of the PATH. It fixed itself when i added a semicolon after the last variable and stayed fixed.

But now to the part where Vaadin comes into play:

But that shouldn't cause this problem, since atleast in the post from calaador up there Vaadin should just download and install node and npm itself when it doesn't find anything.

Unfortunately it seems something changed since then, since FrontendTools.getNpmExecutable() just doesn't care that no npm exists and builds together a faulty command. But it seems ´getNodeExecutable` does try to install these, but it is never called here.

Side note: TaskRunNpmInstall.runNpmInstall() and the methods in FrontEndTools it calls are completely devoid of any logging, might be good to put some in there.

Side note2: I have been hanging on this line for the last 12minutes:

2021-11-12 15:41:21.670  INFO 14352 --- [           main] dev-updater                              : Running `npm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...

I suspect it's because of our corporate proxy, but i can't be sure because: no logging ;)

[caalador]

So basically: One issue is that Node was found(and thus nothing installed) but then finding of npm/nvm failed and there is some place that misses that no result was returned for the command.

For the second one for the proxy I don't think we can notice it (perhaps the timeout time could be shorter on that), and for the missing logging setting the log level to debug should give anything that the npm install task would log. The logging is complex as either you get too much or not enough, but we have tried to have the output available at either debug or trace level.

[wlfbck]

One issue is that Node was found(and thus nothing installed)

Shouldn't be the case, since the bug with the environment variables affects all 3 at once. I think node is simply not checked at all (or later on, but obv i'm never reaching that point).

and for the missing logging setting the log level to debug should give anything that the npm install task would log.

Ah no i meant inside the Vaadin code, there is no logging like "could not find xyz" there is just nothing.

The logging is complex as either you get too much or not enough,

I can relate to that ;)