Building lumo fails on Arch

[futuro]

Running boot release fails while packaging up the first stage binary, specifically task package-executable. This fails because node requires the python executable to point to python 2.6 or 2.7, but on arch the default python is 3.x. From the nodejs-git aur PKGBUILD the following find + sed combo fixes all the references to python.

  find -type f -exec sed \
    -e 's_^#!/usr/bin/env python$_&2_' \
    -e 's_^\(#!/usr/bin/python2\).[45]$_\1_' \
    -e 's_^#!/usr/bin/python$_&2_' \
    -e 's_^\( *exec \+\)python\( \+.*\)$_\1python2\2_'\
    -e 's_^\(.*\)python\( \+-c \+.*\)$_\1python2\2_'\
    -e "s_'python'_'python2'_" -i {} \;
  find test/ -type f -exec sed 's_python _python2 _' -i {} \;

Another approach could be to link the python2 binary into a sub-path of the project during build, such as the tmp directory that's created to hold the node source for nexe, then prepend the PATH directory with that path.

ln -s /usr/bin/python2 tmp/python
PATH=$PWD/tmp:$PATH
boot release

Honestly, I don't particularly like either of these options, but the latter avoids issues around failing to replace every call to python.

[anmonteiro]

Nexe has a python option to which we can pass the path to a python executable. Unfortunately my first attempt at specifying /usr/bin/python2.7 also fails the build process. Can you try that on your end?

[futuro]

I think the issue is that there are lots of calls to python that look like they might call python from the environment, though I'm not positive as I don't know gyp.

For example, when I set the python option, I get the following output

/usr/bin/python2 tools/gyp_node.py -f make
  File "<string>", line 1
    import sys; print sys.byteorder
                        ^
SyntaxError: invalid syntax
gyp: Call to 'python -c "import sys; print sys.byteorder"' returned exit status 1 while in /home/futuro/devel/clojure/lumo/tmp/node/7.2.0/node-v7.2.0/deps/v8/src/v8.gyp.
make: *** [Makefile:79: out/Makefile] Error 1
....> ERROR: The release executable has not been generated. This indicates a failure in the build process. There is likely additional information above.
clojure.lang.ExceptionInfo: node: non-zero exit status (1)
    data: {:file "/tmp/boot.user121522618863490297.clj", :line 53}
       java.lang.Exception: node: non-zero exit status (1)
                     boot.util/dosh                         util.clj:  403
                                ...
        boot.user/eval1365/fn/fn/fn  boot.user121522618863490297.clj:   39
        boot.user/eval1482/fn/fn/fn  boot.user121522618863490297.clj:   45
        boot.user/eval1404/fn/fn/fn  boot.user121522618863490297.clj:   41
        boot.user/eval1326/fn/fn/fn  boot.user121522618863490297.clj:   37
        boot.user/eval1090/fn/fn/fn  boot.user121522618863490297.clj:   23
     boot.task.built-in/fn/fn/fn/fn                     built_in.clj:  303
     boot.task.built-in/fn/fn/fn/fn                     built_in.clj:  493
        boot.user/eval1175/fn/fn/fn  boot.user121522618863490297.clj:   29
        boot.user/eval1132/fn/fn/fn  boot.user121522618863490297.clj:   27
     boot.task.built-in/fn/fn/fn/fn                     built_in.clj:  493
     boot.task.built-in/fn/fn/fn/fn                     built_in.clj:  493
     boot.task.built-in/fn/fn/fn/fn                     built_in.clj:  493
  adzerk.boot-cljs/eval311/fn/fn/fn                    boot_cljs.clj:  208
  adzerk.boot-cljs/eval257/fn/fn/fn                    boot_cljs.clj:  135
        boot.user/eval1050/fn/fn/fn  boot.user121522618863490297.clj:   21
     boot.task.built-in/fn/fn/fn/fn                     built_in.clj:  167
                boot.core/run-tasks                         core.clj:  938
                  boot.core/boot/fn                         core.clj:  948
clojure.core/binding-conveyor-fn/fn                         core.clj: 2020
                                ...

And when I grep for import sys; print sys.byteorder I get the following

node/7.2.0/node-v7.2.0/deps/v8/gypfiles/toolchain.gypi:43: 'v8_host_byteorder%': '<!(python -c "import sys; print sys.byteorder")',

That particular gyp syntax, <!() is a call out to a shell function, as defined here https://gyp.gsrc.io/docs/InputFormatReference.md#Command-Expansions

Since that call is going to ping the PATH to find python, I'm not really certain how nexe would solve that issue; it seems like the kind of thing the NodeJS project would have to support itself. Otherwise, symlinking python2 as python at the front of your path does solve this issue.

[anmonteiro]

A user reported being able to build Lumo on Arch with the following solution, which may not be desirable at all.

ln -s /usr/local/bin/python /usr/bin/python2

[futuro]

To avoid suddenly running every python script on your machine with python2 (when they're written for python3), I think the following set of commands is probably the cleanest solution until nexe/gyp/et. al. move over to python3

cd <lumo dir>
mkdir tmp
ln -s /usr/bin/python2 tmp/python
PATH=$PWD/tmp:$PATH
boot release

It's essentially the same as linking into /usr/local/bin, but instead links into the lumo project directory and temporarily changes your path to start with that tmp directory.

I'm not sure if it's advisable to adopt this fix in lumo, but it would be nice for it to detect when python 3 is the default python and use the above fix for the build process.

[rpodgorny]

@futuro thanks for the tip! ...i've hacked a quick and ugly pkgbuild here:

https://aur.archlinux.org/packages/lumo-git/

[stardiviner]

I got a problem too:

/home/stardiviner/Libraries/node_modules/bin/lumo -> /home/stardiviner/Libraries/node_modules/lib/node_modules/lumo-cljs/bin/lumo.js

> [email protected] install /home/stardiviner/Libraries/node_modules/lib/node_modules/lumo-cljs
> node scripts/npm_install.js || nodejs scripts/npm_install.js


Download failed.
sh: nodejs: command not found
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! [email protected] install: `node scripts/npm_install.js || nodejs scripts/npm_install.js`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/stardiviner/.npm/_logs/2018-03-18T09_47_22_368Z-debug.log

Arch Linux System

I have Node.js installed, don't know why it try to use command nodejs. I tried to create alias for alias nodejs=node. But still failed.

[rpodgorny]

just to let you know - i've just sucessfully built lumo-git package on arch.

[arichiardi]

Also FYI, I built a docker image for building it with static linking here: https://github.com/arichiardi/docker-lumo-musl

In case it is helpful.