node-packer
node-packer copied to clipboard
Published
20 hours ago •
pmq20
pmq20
Configure step fails in general
On Linux using nodec 1.4.0, I ran into this problem when trying to compile the web application example:
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> Running ["./configure "]
Please use either Python 2.6 or 2.7:
/usr/bin/python2 ./configure
Failed running ["./configure "]
It also happen for --clean-tmpdir. Got any ideas? I have searched for the existing issues, but it seems new.
Complete log
(Click to expand)
[znk@crunch helloworld]$ nodec --skip-npm-install node_modules/.bin/egg-bin
Node.js Compiler (nodec) v1.4.0
- entrance: /home/znk/Workspace/examples/helloworld/node_modules/.bin/egg-bin
- options: {:skip_npm_install=>true, :npm=>"npm", :make_args=>"-j4", :vcbuild_args=>"x64", :output=>"/home/znk/Workspace/examples/helloworld/a.out", :tmpdir=>"/tmp/nodec"}
=== WARNING ===
Please make sure to have installed the correct version of node in your environment.
It should match the enclosed Node.js runtime version of the compiler.
Expecting v8.2.0; yet got v8.2.1.
-> mkdir -p /tmp/nodec
-> rm -rf /tmp/nodec/__work_dir__
-> mkdir -p /tmp/nodec/__work_dir__
-> cp -r "/home/znk/Workspace/examples/helloworld" "/tmp/nodec/__work_dir__/__enclose_io_memfs__"
-> cd /tmp/nodec/__work_dir__/__enclose_io_memfs__
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> rm -f deps/libsquash/sample/enclose_io_memfs.squashfs
-> rm -f deps/libsquash/sample/enclose_io_memfs.c
-> Running ["mksquashfs -version"]
mksquashfs version 4.3 (2014/05/12)
copyright (C) 2014 Phillip Lougher <[email protected]>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2,
or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
-> Running ["mksquashfs /tmp/nodec/__work_dir__ deps/libsquash/sample/enclose_io_memfs.squashfs"]
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on deps/libsquash/sample/enclose_io_memfs.squashfs, block size 131072.
[=======================================================================================/] 6214/6214 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 6239.55 Kbytes (6.09 Mbytes)
21.77% of uncompressed filesystem size (28665.40 Kbytes)
Inode table size 66537 bytes (64.98 Kbytes)
27.65% of uncompressed inode table size (240622 bytes)
Directory table size 63401 bytes (61.92 Kbytes)
40.74% of uncompressed directory table size (155624 bytes)
Number of duplicate files found 1174
Number of inodes 7453
Number of files 6187
Number of fragments 138
Number of symbolic links 77
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1189
Number of ids (unique uids + gids) 2
Number of uids 1
znk (1000)
Number of gids 1
users (100)
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> Running ["./configure "]
Please use either Python 2.6 or 2.7:
/usr/bin/python2 ./configure
Failed running ["./configure "]
Log with --debug param
(Click to expand)
[znk@crunch helloworld]$ nodec --debug --skip-npm-install node_modules/.bin/egg-bin
Node.js Compiler (nodec) v1.4.0
- entrance: /home/znk/Workspace/examples/helloworld/node_modules/.bin/egg-bin
- options: {:debug=>true, :skip_npm_install=>true, :npm=>"npm", :make_args=>"-j4", :vcbuild_args=>"x64", :output=>"/home/znk/Workspace/examples/helloworld/a.out", :tmpdir=>"/tmp/nodec"}
=== WARNING ===
Please make sure to have installed the correct version of node in your environment.
It should match the enclosed Node.js runtime version of the compiler.
Expecting v8.2.0; yet got v8.2.1.
-> mkdir -p /tmp/nodec
-> rm -rf /tmp/nodec/__work_dir__
-> mkdir -p /tmp/nodec/__work_dir__
-> cp -r "/home/znk/Workspace/examples/helloworld" "/tmp/nodec/__work_dir__/__enclose_io_memfs__"
-> cd /tmp/nodec/__work_dir__/__enclose_io_memfs__
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> rm -f deps/libsquash/sample/enclose_io_memfs.squashfs
-> rm -f deps/libsquash/sample/enclose_io_memfs.c
-> Running ["mksquashfs -version"]
mksquashfs version 4.3 (2014/05/12)
copyright (C) 2014 Phillip Lougher <[email protected]>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2,
or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
-> Running ["mksquashfs /tmp/nodec/__work_dir__ deps/libsquash/sample/enclose_io_memfs.squashfs"]
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on deps/libsquash/sample/enclose_io_memfs.squashfs, block size 131072.
[=======================================================================================/] 6214/6214 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 6239.97 Kbytes (6.09 Mbytes)
21.77% of uncompressed filesystem size (28665.40 Kbytes)
Inode table size 66960 bytes (65.39 Kbytes)
27.83% of uncompressed inode table size (240622 bytes)
Directory table size 63403 bytes (61.92 Kbytes)
40.74% of uncompressed directory table size (155624 bytes)
Number of duplicate files found 1174
Number of inodes 7453
Number of files 6187
Number of fragments 138
Number of symbolic links 77
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1189
Number of ids (unique uids + gids) 2
Number of uids 1
znk (1000)
Number of gids 1
users (100)
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> cd /home/znk/Workspace/examples/helloworld
-> cd /tmp/nodec/node-8.2.0-1.4.0
-> Running ["./configure --debug"]
Please use either Python 2.6 or 2.7:
/usr/bin/python2 ./configure --debug
Failed running ["./configure --debug"]
System infos
(Click to expand)
[znk@crunch ~]$ uname -a
Linux crunch 4.12.3-1-ARCH #1 SMP PREEMPT Sat Jul 22 15:32:02 UTC 2017 x86_64 GNU/Linux
[znk@crunch ~]$ lsb_release -a
LSB Version: 1.4
Distributor ID: Arch
Description: Arch Linux
Release: rolling
Codename: n/a
[znk@crunch ~]$ sudo df -h
Filesystem Size Used Avail Use% Mounted on
dev 3.9G 0 3.9G 0% /dev
run 3.9G 984K 3.9G 1% /run
/dev/sda3 29G 11G 17G 40% /
tmpfs 3.9G 442M 3.5G 12% /dev/shm
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
tmpfs 3.9G 1.5M 3.9G 1% /etc/pacman.d/gnupg
/dev/sda4 880G 35G 800G 5% /home
tmpfs 3.9G 145M 3.8G 4% /tmp
/dev/sda1 197M 48M 150M 24% /boot
tmpfs 788M 28K 788M 1% /run/user/1000
Requisites
(Click to expand)
-
Nodec
[znk@crunch ~]$ whereis nodec
nodec: /usr/local/bin/nodec
[znk@crunch ~]$ nodec -v
1.4.0
-
SquashFS Tools 4.3
[znk@crunch ~]$ pacman -Qi squashfs-tools | grep -E "Name|Version|Architecture"
Name : squashfs-tools
Version : 4.3-4
Architecture : x86_64
-
gcc and g++ 4.9.4 or newer
[znk@crunch ~]$ pacman -Qi gcc | grep -E "Name|Version|Architecture"
Name : gcc
Version : 7.1.1-4
Architecture : x86_64
-
clang and clang++ 3.4.2 or newer
[znk@crunch ~]$ pacman -Qi clang | grep -E "Name|Version|Architecture"
Name : clang
Version : 4.0.1-5
Architecture : x86_64
-
Python 2.6 or 2.7
[znk@crunch ~]$ pacman -Qi python2 | grep -E "Name|Version|Architecture"
Name : python2
Version : 2.7.13-4
Architecture : x86_64
-
GNU Make 3.81 or newer
[znk@crunch ~]$ pacman -Qi make | grep -E "Name|Version|Architecture"
Name : make
Version : 4.2.1-2
Architecture : x86_64
-
Node
[znk@crunch ~]$ pacman -Qi nodejs | grep -E "Name|Version|Architecture"
Name : nodejs
Version : 8.2.1-1
Architecture : x86_64
-
NPM
[znk@crunch ~]$ pacman -Qi npm | grep -E "Name|Version|Architecture"
Name : npm
Version : 5.3.0-1
Architecture : any
-
Yarn
[znk@crunch ~]$ pacman -Qi yarn | grep -E "Name|Version|Architecture"
Name : yarn
Version : 0.28.4-1
Architecture : any
Investigation
So node-compiler is a thin ruby wrapper around multiple big projects. :open_mouth: . Issue has a big chance to be located somewhere else. Getting in /tmp/nodec/, there is two occurrences of Python' configure :
[znk@crunch nodec]$ find -name "configure"
./node-8.2.0-1.4.0/deps/zlib/configure
./node-8.2.0-1.4.0/configure
zlib' configure is a simple bash script. Whereas node' configure is big python script, and automatically generated. Here is its content. Lucky me, it fails at the very start:
#!/usr/bin/env python
import sys
if sys.version_info[0] != 2 or sys.version_info[1] not in (6, 7):
sys.stderr.write('Please use either Python 2.6 or 2.7')
from distutils.spawn import find_executable as which
python2 = which('python2') or which('python2.6') or which('python2.7')
if python2:
sys.stderr.write(':\n\n')
sys.stderr.write(' ' + python2 + ' ' + ' '.join(sys.argv))
sys.stderr.write('\n')
sys.exit(1)
It seems that this generated script does not locate correctly the installed instances of Python.
[znk@crunch ~]$ ls -la /usr/bin/python*
lrwxrwxrwx 1 root root 7 Jul 19 21:53 /usr/bin/python -> python3
lrwxrwxrwx 1 root root 9 Jul 20 21:25 /usr/bin/python2 -> python2.7
-rwxr-xr-x 1 root root 6216 Jul 20 21:25 /usr/bin/python2.7
-rwxr-xr-x 1 root root 1681 Jul 20 21:25 /usr/bin/python2.7-config
lrwxrwxrwx 1 root root 16 Jul 20 21:25 /usr/bin/python2-config -> python2.7-config
lrwxrwxrwx 1 root root 9 Jul 19 21:53 /usr/bin/python3 -> python3.6
-rwxr-xr-x 2 root root 10368 Jul 19 21:54 /usr/bin/python3.6
lrwxrwxrwx 1 root root 17 Jul 19 21:53 /usr/bin/python3.6-config -> python3.6m-config
-rwxr-xr-x 2 root root 10368 Jul 19 21:54 /usr/bin/python3.6m
-rwxr-xr-x 1 root root 3146 Jul 19 21:53 /usr/bin/python3.6m-config
lrwxrwxrwx 1 root root 16 Jul 19 21:53 /usr/bin/python3-config -> python3.6-config
lrwxrwxrwx 1 root root 14 Jul 19 21:53 /usr/bin/python-config -> python3-config
I will investigate further ~~symlinking~~ or putting an exception to node-gyp' python detection.
(click to expand)
[znk@crunch node-8.2.0-1.4.0]$ python configure
Please use either Python 2.6 or 2.7:
/usr/bin/python2 configure
[znk@crunch node-8.2.0-1.4.0]$ python2 configure
creating icu_config.gypi
* Using ICU in deps/icu-small
creating icu_config.gypi
{ 'target_defaults': { 'cflags': [],
'default_configuration': 'Release',
'defines': [],
'include_dirs': [],
'libraries': []},
'variables': { 'asan': 0,
'coverage': 'false',
'debug_devtools': 'node',
'force_dynamic_crt': 0,
'gas_version': '2.28',
'host_arch': 'x64',
'icu_data_file': 'icudt59l.dat',
'icu_data_in': '../../deps/icu-small/source/data/in/icudt59l.dat',
'icu_endianness': 'l',
'icu_gyp_path': 'tools/icu/icu-generic.gyp',
'icu_locales': 'en,root',
'icu_path': 'deps/icu-small',
'icu_small': 'true',
'icu_ver_major': '59',
'node_byteorder': 'little',
'node_enable_d8': 'false',
'node_enable_v8_vtunejit': 'false',
'node_install_npm': 'true',
'node_module_version': 57,
'node_no_browser_globals': 'false',
'node_prefix': '/usr/local',
'node_release_urlbase': '',
'node_shared': 'false',
'node_shared_cares': 'false',
'node_shared_http_parser': 'false',
'node_shared_libuv': 'false',
'node_shared_openssl': 'false',
'node_shared_zlib': 'false',
'node_tag': '',
'node_use_bundled_v8': 'true',
'node_use_dtrace': 'false',
'node_use_etw': 'false',
'node_use_lttng': 'false',
'node_use_openssl': 'true',
'node_use_perfctr': 'false',
'node_use_v8_platform': 'true',
'node_without_node_options': 'false',
'openssl_fips': '',
'openssl_no_asm': 0,
'shlib_suffix': 'so.57',
'target_arch': 'x64',
'uv_parent_path': '/deps/uv/',
'uv_use_dtrace': 'false',
'v8_enable_gdbjit': 0,
'v8_enable_i18n_support': 1,
'v8_enable_inspector': 1,
'v8_no_strict_aliasing': 1,
'v8_optimized_debug': 0,
'v8_promise_internal_field_count': 1,
'v8_random_seed': 0,
'v8_trace_maps': 0,
'v8_use_snapshot': 'false',
'want_separate_host_toolset': 0,
'want_separate_host_toolset_mkpeephole': 0}}
creating config.gypi
creating config.mk
File "<string>", line 1
import sys; print sys.byteorder
^
SyntaxError: invalid syntax
gyp: Call to 'python -c "import sys; print sys.byteorder"' returned exit status 1 while in /tmp/nodec/node-8.2.0-1.4.0/deps/v8/src/v8.gyp. while loading dependencies of /tmp/nodec/node-8.2.0-1.4.0/node.gyp while trying to load /tmp/nodec/node-8.2.0-1.4.0/node.gyp
Error running GYP
Further investigation
So, the root cause is node-gyp being stuck in Python 2 whereas his generated script is executed with my system' default python, alias to python3.
Quoting Gibson Fahnestock (2016-11-30 17:34:41)
@silverwind I think the problem is that if python points to Python 3, then even if you run the main configure file with ./python2 configure, when subshells are spawned with the #!/usr/bin/env python, then they end up still running with python3, which doesn't work. Exactly. On Archlinux, /usr/bin/python is python 3. Putting something that points to python 2 ahead of it in $PATH would do the trick.
Workaround as of August 1, 2017 on ArchLinux
mkdir /home/$USER/python2
ln -s /usr/bin/python2 /home/$USER/python2/python
git clone --depth 1 https://github.com/eggjs/examples.git
cd examples/helloworld
npm install
PATH=/home/$USER/python2:$PATH nodec node_modules/egg-bin/bin/egg-bin.js
./a.out dev
It also applies to other params as well: PATH=/home/$USER/python2:$PATH nodec --clean-tmpdir
(click to expand)
if [ ! -r node -o ! -L node ]; then ln -fs out/Release/node node; fi
-> cd /home/znk/Workspace/examples/helloworld
-> cp "/tmp/nodec/node-8.2.0-1.4.0/out/Release/node" "/home/znk/Workspace/examples/helloworld/a.out"
[znk@crunch helloworld]$ ./a
a.out app/
[znk@crunch helloworld]$ ./a
a.out app/
[znk@crunch helloworld]$ ./a.out
Usage: [command] [options]
Options:
-V, --version output the version number
-h, --help output usage information
Commands:
* See "More commands"
More commands
dev - local env start
debug - Debug mode start
test - Run test with mocha
cov - Run test with coverage
Hello @pmq20, since this case seems specific to some Linux distributions, I will leave those posts here for future references. How do you handle platform specific scripts in Nodec? I saw --make-args param, but my attempt to make use of it has not been successful. Mind to provide an example? Also, to which Makefile it refers to?
I will package nodec soon for Arch.
@znk Thanks for the detailed investigation! Seems like we should file a patch to node-gyp to let it try the command python2 in case of python being linked to python3.
--make-args is for arguments passed to make when executing against node/Makefile. An example would be: nodec --make-args="-j4 V=1".
