node
node copied to clipboard
nodejs
build: fix building when there is only python3
On some environments, especially macOS, there is only python3 and no python, replacing the python to <(python) can make gyp use the correct python.
The error is:
gyp: Undefined variable python in /home/runner/work/node/node/tools/v8_gypfiles/v8.gyp while loading dependencies of /home/runner/work/node/node/node.gyp while trying to load /home/runner/work/node/node/node.gyp
Failed to start CI
- Validating Jenkins credentials ✘ Jenkins credentials invalidhttps://github.com/nodejs/node/actions/runs/5295843320
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Failed to start CI
- Validating Jenkins credentials ✘ Jenkins credentials invalidhttps://github.com/nodejs/node/actions/runs/5298312237
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=benchmark-ubuntu2204-intel-64,v8test=v8test/5409/
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=rhel8-s390x,v8test=v8test/5409/
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=rhel8-ppc64le,v8test=v8test/5409/
broken on windows arm64 cc @nodejs/platform-arm @StefanStojanovic
11:04:03 'C:\Python310\python_host.exe' is not recognized as an internal or external command,
11:04:03 operable program or batch file.
11:04:03 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'out\Release\\node_js2c_host.exe;src\inspector\node_protocol.pdl;src\inspector\node_protocol_config.json;deps\v8\include\js_protocol.pdl;out\Release\\obj\global_intermediate\concatenated_protocol.json;deps\openssl\openssl\util\libcrypto.num' exited with code 1. [C:\workspace\node-compile-windows\node\libnode.vcxproj]
11:04:03 gen-regexp-special-case.cc
11:04:03 C:\workspace\node-compile-windows\node\deps\v8\src\base\bits.h(448,31): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\gen-regexp-special-case_host.vcxproj]
11:04:04 Creating library ..\..\out\Release\gen-regexp-special-case_host.lib and object ..\..\out\Release\gen-regexp-special-case_host.exp
11:04:04 gen-regexp-special-case_host.vcxproj -> ..\..\out\Release\\gen-regexp-special-case_host.exe
11:04:05 run_gen-regexp-special-case_action
11:04:05 'C:\Python310\python_host.exe' is not recognized as an internal or external command,
11:04:05 operable program or batch file.
11:04:05 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for '..\..\out\Release\\gen-regexp-special-case_host.exe' exited with code 1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\run_gen-regexp-special-case.vcxproj]
11:04:05 run_gen-regexp-special-case_action
11:04:05 'C:\Python310\python_host.exe' is not recognized as an internal or external command,
11:04:05 operable program or batch file.
11:04:05 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for '..\..\out\Release\\gen-regexp-special-case_host.exe' exited with code 1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\run_gen-regexp-special-case_host.vcxproj]
11:04:05
11:04:05 > if errorlevel 1 (
11:04:05 if not defined project_generated echo Building Node with reused solution failed. To regenerate project files use "vcbuild projgen"
broken on windows arm64 cc @nodejs/platform-arm @StefanStojanovic
11:04:03 'C:\Python310\python_host.exe' is not recognized as an internal or external command, 11:04:03 operable program or batch file. 11:04:03 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'out\Release\\node_js2c_host.exe;src\inspector\node_protocol.pdl;src\inspector\node_protocol_config.json;deps\v8\include\js_protocol.pdl;out\Release\\obj\global_intermediate\concatenated_protocol.json;deps\openssl\openssl\util\libcrypto.num' exited with code 1. [C:\workspace\node-compile-windows\node\libnode.vcxproj] 11:04:03 gen-regexp-special-case.cc 11:04:03 C:\workspace\node-compile-windows\node\deps\v8\src\base\bits.h(448,31): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\gen-regexp-special-case_host.vcxproj] 11:04:04 Creating library ..\..\out\Release\gen-regexp-special-case_host.lib and object ..\..\out\Release\gen-regexp-special-case_host.exp 11:04:04 gen-regexp-special-case_host.vcxproj -> ..\..\out\Release\\gen-regexp-special-case_host.exe 11:04:05 run_gen-regexp-special-case_action 11:04:05 'C:\Python310\python_host.exe' is not recognized as an internal or external command, 11:04:05 operable program or batch file. 11:04:05 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for '..\..\out\Release\\gen-regexp-special-case_host.exe' exited with code 1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\run_gen-regexp-special-case.vcxproj] 11:04:05 run_gen-regexp-special-case_action 11:04:05 'C:\Python310\python_host.exe' is not recognized as an internal or external command, 11:04:05 operable program or batch file. 11:04:05 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for '..\..\out\Release\\gen-regexp-special-case_host.exe' exited with code 1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\run_gen-regexp-special-case_host.vcxproj] 11:04:05 11:04:05 > if errorlevel 1 ( 11:04:05 if not defined project_generated echo Building Node with reused solution failed. To regenerate project files use "vcbuild projgen"
Friendly ping
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=benchmark-ubuntu2204-intel-64,v8test=v8test/5495/
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=rhel8-s390x,v8test=v8test/5495/
V8 CI: https://ci.nodejs.org/job/node-test-commit-v8-linux/nodes=rhel8-ppc64le,v8test=v8test/5495/
broken on windows arm64 cc @nodejs/platform-arm @StefanStojanovic
11:04:03 'C:\Python310\python_host.exe' is not recognized as an internal or external command, 11:04:03 operable program or batch file. 11:04:03 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'out\Release\\node_js2c_host.exe;src\inspector\node_protocol.pdl;src\inspector\node_protocol_config.json;deps\v8\include\js_protocol.pdl;out\Release\\obj\global_intermediate\concatenated_protocol.json;deps\openssl\openssl\util\libcrypto.num' exited with code 1. [C:\workspace\node-compile-windows\node\libnode.vcxproj] 11:04:03 gen-regexp-special-case.cc 11:04:03 C:\workspace\node-compile-windows\node\deps\v8\src\base\bits.h(448,31): warning C4146: unary minus operator applied to unsigned type, result still unsigned [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\gen-regexp-special-case_host.vcxproj] 11:04:04 Creating library ..\..\out\Release\gen-regexp-special-case_host.lib and object ..\..\out\Release\gen-regexp-special-case_host.exp 11:04:04 gen-regexp-special-case_host.vcxproj -> ..\..\out\Release\\gen-regexp-special-case_host.exe 11:04:05 run_gen-regexp-special-case_action 11:04:05 'C:\Python310\python_host.exe' is not recognized as an internal or external command, 11:04:05 operable program or batch file. 11:04:05 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for '..\..\out\Release\\gen-regexp-special-case_host.exe' exited with code 1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\run_gen-regexp-special-case.vcxproj] 11:04:05 run_gen-regexp-special-case_action 11:04:05 'C:\Python310\python_host.exe' is not recognized as an internal or external command, 11:04:05 operable program or batch file. 11:04:05 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for '..\..\out\Release\\gen-regexp-special-case_host.exe' exited with code 1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\run_gen-regexp-special-case_host.vcxproj] 11:04:05 11:04:05 > if errorlevel 1 ( 11:04:05 if not defined project_generated echo Building Node with reused solution failed. To regenerate project files use "vcbuild projgen"Friendly ping
since skip windows fix looks need extra work, can you help with the windows arm team @mhdawson
@nodejs/tsc Seems windows-arm not responding, can you help on this one. This has been a huge pain on newer linux and macOS (only python3 is available)
@nodejs/tsc Seems windows-arm not responding, can you help on this one.
You want to be pinging @nodejs/platform-windows-arm for Windows on ARM.
@StefanStojanovic and @joaocgreis are maintaining this, hopefully they'll come to help on this.
@nodejs/tsc Seems windows-arm not responding, can you help on this one.
You want to be pinging @nodejs/platform-windows-arm for Windows on ARM.
Looks windows-arm not responding at all. @nodejs/tsc Can we help on this one ?
I pingned the people who initially wanted to get Windows on arm into the CIs through email but not sure if that will have any effect.
We are sorry to hear you're having issue with Windows on Arm support, we worked on (partially) at Linaro. Let's hope MS can find a solution for this.
@zcbenz, since this branch is 500+ commits behind the main branch, can you please rebase it before I start looking into the Windows on ARM64 issue?
@zcbenz I've started looking into your changes. Here is the first problem I came across - although compiling x64/x86 passes in CI, for me locally it doesn't work. Here is my log:
E:\work\node> .\vcbuild.bat
Looking for Python
Python found in C:\Program Files\Python310\\python.exe
Python 3.10.11
Looking for NASM
Looking for Visual Studio 2022
calling: "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\\Auxiliary\Build\vcvarsall.bat" amd64
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.7.4
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
Found MSVS version 17.0
configure --dest-cpu=x64
Node.js configure: Found Python 3.10.11...
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'C:\Program Files\Python310\python.exe -c "import sys; print(sys.byteorder)"' returned exit status 1 while in E:\work\node\tools\v8_gypfiles\v8.gyp. while loading dependencies of E:\work\node\node.gyp while trying to load E:\work\node\node.gyp
Error running GYP
Failed to create vc project files.
For me, Python is found at C:\Program Files\Python310\, which is the default location for installing Python 3.10 via the installer. In the CI, we install Python via Chocolatey, so it's at C:\Python310\, and as there are no spaces in that path it works. (here is a totally unrelated issue that mentions the difference between installing Python via the installer and Chocolatey). Anyway, I believe, most people use the installer, rather than Chocolatey, and we should certainly make sure that even if the path to Python has spaces, it still works for compiling Node eg. by adding quotes around it.
The only way I was able to at least generate project files (running vcbuild.bat projgen nobuild) was by reverting https://github.com/nodejs/node/pull/48462/commits/1a120f356584022c1e4e08e93178e22dba2ef2fc. After I've done that, I saw that C:\Program Files\Python310\python_host.exe was used for custom build steps on arm64 instead of C:\Program Files\Python310\python.exe which was used for x64/x86. Just for reference, on the main branch, in all 3 cases, I only saw python.
This is all I've got for now. Next week I'll be OOF so, I will only be able to pick this up again from October 30, but I wanted to at least share my initial findings as they might be helpful to someone.
Thanks for looking into it, I am able to reproduce the same error with this setup.