Compile error with MacPorts' sbcl

[jrjsmrtn]

I'm trying to install common-lisp-jupyter on macOS 10.14.4 with MacPorts 2.5.4 but got this:

$ sbcl --noinform --eval '(ql:update-dist "quicklisp")' --quit
You already have the latest version of "quicklisp": 2019-03-07.

$ sbcl --noinform --eval '(ql:quickload :common-lisp-jupyter)' --quit
To load "common-lisp-jupyter":
  Load 1 ASDF system:
    common-lisp-jupyter
; Loading "common-lisp-jupyter"
...; /usr/bin/clang -o /Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel-tmpGHU3ALSV.o -c -g -Wall -Wundef -Wsign-compare -Wpointer-arith -O3 -g -Wall -O2 -fdollars-in-identifiers -mmacosx-version-min=10.14 -D_DARWIN_USE_64_BIT_INODE -arch x86_64 -fno-omit-frame-pointer -fPIC -I/Users/gm/quicklisp/dists/quicklisp/software/cffi_0.20.0/ /Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel.c
./Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel.c:6:10: fatal error: 'zmq.h' file not found
#include <zmq.h>
         ^~~~~~~
1 error generated.

debugger invoked on a CFFI-GROVEL:GROVEL-ERROR in thread #<THREAD "main thread" RUNNING {10005C85B3}>: Subprocess #<UIOP/LAUNCH-PROGRAM::PROCESS-INFO {1004847CF3}>
 with command ("/usr/bin/clang" "-o" "/Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel-tmpGHU3ALSV.o" "-c" "-g" "-Wall" "-Wundef" "-Wsign-compare" "-Wpointer-arith" "-O3" "-g" "-Wall" ...)
 exited with error code 1

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

ZeroMQ 4.3.1 is installed through MacPorts and its headers are in /opt/local/include:

$ port installed | grep zmq
  py36-zmq @18.0.1_0 (active)
  zmq @4.3.1_0 (active)

$ ls /opt/local/include/zmq*
/opt/local/include/zmq.h       /opt/local/include/zmq_utils.h

A quick, naïve look into cffi seems to indicate that it knows about MacPorts' /opt/local/include directory:

quicklisp/dists/quicklisp/software/cffi_0.20.0/libffi/libffi-types.lisp
32-
33-;; When installed through Mac Ports, libffi include files
34:;; will be found in /opt/local/include.
35-#+darwin
36:(cc-flags "-I/opt/local/include/")
37-
38-#+openbsd

quicklisp/dists/quicklisp/software/cffi_0.20.0/toolchain/c-toolchain.lisp
210-           arch-flags
211-           ;; For MacPorts
212:           #+darwin (list "-I" "/opt/local/include/")
213-           ;; ECL internal flags
214-           #+ecl (parse-command-flags c::*cc-flags*)

Any suggestion?

[yitzchak]

I have very little knowledge about Macs let alone MacPorts. I've only successfully installed zmq using homebrew. I'll try to do some investigation, but it may take a bit since I don't have any experience with MacPorts. In the meantime I'll mark as "help wanted" in case some one else has some ideas.

[jrjsmrtn]

Thanks.

FYI, I manually added the /opt/local/include directory here and grovel.c compiled:

$ /usr/bin/clang \
    -o /Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel-tmpGHU3ALSV.o \
    -c -g -Wall -Wundef -Wsign-compare -Wpointer-arith -O3 -g -Wall \
    -O2 -fdollars-in-identifiers -mmacosx-version-min=10.14 \
    -D_DARWIN_USE_64_BIT_INODE -arch x86_64 -fno-omit-frame-pointer -fPIC \
    -I/Users/gm/quicklisp/dists/quicklisp/software/cffi_0.20.0/ \
    -I/opt/local/include/ \
    /Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel.c

/Users/gm/.cache/common-lisp/sbcl-1.5.0-macosx-x64/Users/gm/quicklisp/dists/quicklisp/software/pzmq-20171019-git/grovel__grovel.c:11:7: warning:
      unused variable 'autotype_tmp' [-Wunused-variable]
  int autotype_tmp;
      ^
1 warning generated.

I'll dig further to understand why /opt/local/include is not passed to the compiler.

[yitzchak]

It doesn't appear that grovel knows about MacPorts and most packages that use grovel appear to pass flags to the compiler directly. I managed to get pzmq to compile by adding cc-flags to the top of grovel.lisp

(in-package #:pzmq)
#+darwin
(cc-flags "-I/opt/local/include/")
(include "zmq.h")

and by adding a :darwin library option in c-api.lisp

(define-foreign-library libzmq
  (:darwin (:or "libzmq.dylib" "/opt/local/lib/libzmq.dylib"))
  (:unix (:or "libzmq.so.3.0.0" "libzmq.so.3" "libzmq"))
  (t (:default "libzmq")))

Once pzmq compiled I was able complete the installation and run the kernel from Jupyter.

I'll submit a PR to pzmq after I investigate a few more CFFI projects and see how they handle MacPorts to make sure that I did it right.

[yitzchak]

This should be fixed by cffi/cffi#139

[jrjsmrtn]

@yitzchak this is not fixed by my CFFI patch. There, I added /opt/local/lib, not /opt/local/include.

I tried again to install common-lisp-jupyter with CFFI 0.20.1 but I still have the same /opt/local/include-related issue.

In the meantime, I am still digging into grovel.