clasp icon indicating copy to clipboard operation
clasp copied to clipboard

Published 20 hours ago • verified publisher icon clasp-developers

Build error: Unable to build clasp from source on M1-Mac (osx-arm64)

Open ludgerpaehler opened this issue 3 years ago • 10 comments
trafficstars

I have recently been trying to compile clasp from source on MacOS with an M1 ARM-chip. While I was able to make it far in the build process, there are 13 files which ninja is unable to compile (error output below).

Is this to be expected at the current time, or have I misconfigured the build?

As far as I am able to tell, most of the errors do seem to originate from the garbage-collection.

Here is one of the build commands:

/Users/lpaehler/miniconda3/envs/clasp-dev/bin/clang -Iboehmprecise -Iboehmprecise/generated -O3 -g -fPIC -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/ -Wno-macro-redefined -Wno-deprecated-declarations -Wno-deprecated-register -Wno-expansion-to-defined -Wno-return-type-c-linkage -Wno-invalid-offsetof -Wno-#pragma-messages -Wno-inconsistent-missing-override -Wno-error=c++11-narrowing -Wno-c++11-narrowing -Wno-deprecated-enum-enum-conversion -Wno-deprecated-anon-enum-enum-conversion -I/usr/local/include -I/Users/lpaehler/miniconda3/envs/clasp-dev/include -I.. -I/opt/homebrew/Cellar/fmt/9.1.0/include -I/opt/homebrew/Cellar/gmp/6.2.1_1/include -I../src/bdwgc -I../src/bdwgc/include -I../src/libatomic_ops/src -I../include -I../include/clasp/main -c -MD -MF boehmprecise/src/gctools/mygc.o.d -oboehmprecise/src/gctools/mygc.o ../src/gctools/mygc.c

And here is the part of the error output - the full error output is attached clasp_build.txt:

In file included from ../src/bdwgc/extra/gc.c:63:
../src/bdwgc/extra/../os_dep.c:4684:32: error: use of undeclared identifier 'x86_THREAD_STATE64'
                               GC_MACH_THREAD_STATE);
                               ^
../src/bdwgc/include/private/gc_priv.h:780:41: note: expanded from macro 'GC_MACH_THREAD_STATE'
#     define GC_MACH_THREAD_STATE       x86_THREAD_STATE64
                                        ^
In file included from ../src/gctools/mygc.c:58:
In file included from ../src/bdwgc/extra/gc.c:67:
../src/bdwgc/extra/../darwin_stop_world.c:166:5: error: unknown type name 'x86_thread_state64_t'; did you mean 'arm_thread_state64_t'?
    GC_THREAD_STATE_T state;
    ^~~~~~~~~~~~~~~~~
    arm_thread_state64_t
../src/bdwgc/include/private/gc_priv.h:779:41: note: expanded from macro 'GC_THREAD_STATE_T'
#     define GC_THREAD_STATE_T          x86_thread_state64_t
                                        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/mach/arm/thread_status.h:102:36: note: 'arm_thread_state64_t' declared here
typedef _STRUCT_ARM_THREAD_STATE64 arm_thread_state64_t;

ludgerpaehler avatar Sep 13 '22 04:09 ludgerpaehler

We don't have M1 support yet. Probably not until LLVM16. You'll have to compile in X86 mode with Rosetta.

yitzchak avatar Sep 13 '22 10:09 yitzchak

Is there any update on this? When I just tried to build on my M1 with LLVM 17.0.3 using ./koga, I get to the "Evaluating cscript files" part, and it chokes here:

While evaluating the form starting at line 185, column 0
  of #P"/Users/rjsheperd/Code/clasp/src/lisp/cscript.lisp":
Unhandled TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                  {70087D0393}>

Complete log:

Evaluating cscript files
Looking for configure scripts in 
Found script cscript.lisp
Loading script cscript.lisp
Looking for configure scripts in include/
Found script include/cscript.lisp
Looking for configure scripts in src/
Found script src/cscript.lisp
Configuring library fmt
Configuring library gmpxx
Loading script src/cscript.lisp
Looking for configure scripts in analysis/
Found script src/analysis/cscript.lisp
Looking for configure scripts in gctools/
Found script src/gctools/cscript.lisp
Looking for configure scripts in clbind/
Found script src/clbind/cscript.lisp
Looking for configure scripts in serveEvent/
Found script src/serveEvent/cscript.lisp
Looking for configure scripts in sockets/
Found script src/sockets/cscript.lisp
Looking for configure scripts in llvmo/
Found script src/llvmo/cscript.lisp
Looking for configure scripts in mpip/
Found script src/mpip/cscript.lisp
Looking for configure scripts in asttooling/
Found script src/asttooling/cscript.lisp
Looking for configure scripts in main/
Looking for configure scripts in main/
Found script src/main/cscript.lisp
Looking for configure scripts in core/
Found script src/core/cscript.lisp
Looking for configure scripts in lisp/
Found script src/lisp/cscript.lisp
Loading script src/lisp/cscript.lisp
While evaluating the form starting at line 185, column 0
  of #P"/Users/rjsheperd/Code/clasp/src/lisp/cscript.lisp":
Unhandled TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                  {70087D0393}>:
  The value
    NIL
  is not of type
    (OR STRING PATHNAME SYNONYM-STREAM FILE-STREAM)
  when binding PATHNAME

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {70087D0393}>
0: (FILE-NAMESTRING NIL) [external]
1: (KOGA::ADD-TARGET-DIRECTORY #<KOGA::CONFIGURATION {70054BD2F3}> :INSTALL-CODE src/lisp/modules/)
2: (KOGA:SOURCES :INSTALL-CODE src/lisp/modules/ src/lisp/kernel/contrib/Acclimation/ src/lisp/kernel/contrib/alexandria/ src/lisp/kernel/contrib/Cleavir/ src/lisp/kernel/contrib/closer-mop/ src/lisp/kernel/contrib/Concrete-Syntax-Tree/ src/lisp/kernel/contrib/Eclector/)
3: (SIMPLE-EVAL-IN-LEXENV (KOGA:SOURCES :INSTALL-CODE src/lisp/modules/ src/lisp/kernel/contrib/Acclimation/ src/lisp/kernel/contrib/alexandria/ src/lisp/kernel/contrib/Cleavir/ src/lisp/kernel/contrib/closer-mop/ src/lisp/kernel/contrib/Concrete-Syntax-Tree/ src/lisp/kernel/contrib/Eclector/) #<NULL-LEXENV>)
4: (EVAL-TLF (KOGA:SOURCES :INSTALL-CODE src/lisp/modules/ src/lisp/kernel/contrib/Acclimation/ src/lisp/kernel/contrib/alexandria/ src/lisp/kernel/contrib/Cleavir/ src/lisp/kernel/contrib/closer-mop/ src/lisp/kernel/contrib/Concrete-Syntax-Tree/ src/lisp/kernel/contrib/Eclector/) 6 NIL)
5: ((LABELS SB-FASL::EVAL-FORM :IN LOAD-AS-SOURCE) (KOGA:SOURCES :INSTALL-CODE src/lisp/modules/ src/lisp/kernel/contrib/Acclimation/ src/lisp/kernel/contrib/alexandria/ src/lisp/kernel/contrib/Cleavir/ src/lisp/kernel/contrib/closer-mop/ src/lisp/kernel/contrib/Concrete-Syntax-Tree/ src/lisp/kernel/contrib/Eclector/) 6)
6: ((LAMBDA (FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN LOAD-AS-SOURCE) (KOGA:SOURCES :INSTALL-CODE src/lisp/modules/ src/lisp/kernel/contrib/Acclimation/ src/lisp/kernel/contrib/alexandria/ src/lisp/kernel/contrib/Cleavir/ src/lisp/kernel/contrib/closer-mop/ src/lisp/kernel/contrib/Concrete-Syntax-Tree/ src/lisp/kernel/contrib/Eclector/) :CURRENT-INDEX 6)
7: (SB-C::%DO-FORMS-FROM-INFO #<FUNCTION (LAMBDA (FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN LOAD-AS-SOURCE) {7006E0301B}> #<SB-C::SOURCE-INFO {7006E02FD3}> SB-C::INPUT-ERROR-IN-LOAD)
8: (LOAD-AS-SOURCE #<FORM-TRACKING-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/src/lisp/cscript.lisp" {7006E017C3}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
9: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<FORM-TRACKING-STREAM for "file /Users/rjsheperd/Code/clasp/src/lisp/cscript.lisp" {7006E017C3}> NIL)
10: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {1010D16BB}> #<FORM-TRACKING-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/src/lisp/cscript.lisp" {7006E017C3}> NIL #<FORM-TRACKING-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/src/lisp/cscript.lisp" {7006E017C3}>)
11: (LOAD "src/lisp/cscript.lisp" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT)
12: (KOGA::DO-SETUP)
13: (KOGA::SETUP :VERSION "2.4.0")
14: ((LAMBDA NIL :IN "/Users/rjsheperd/Code/rjsheperd/clasp/koga"))
15: (SIMPLE-EVAL-IN-LEXENV (LET* ((INITARGS (NCONC (PARSE-COMMAND-LINE-ARGUMENTS) (IGNORE-ERRORS #) (IGNORE-ERRORS #))) (*GIT* (GETF INITARGS :GIT "git")) (BUILD (UIOP/PATHNAME:ENSURE-DIRECTORY-PATHNAME (GETF INITARGS :BUILD-PATH "build/"))) (EXTENSIONS (GETF INITARGS :EXTENSIONS)) (SKIP-SYNC (GETF INITARGS :SKIP-SYNC)) (HELP (GETF INITARGS :HELP))) (WHEN (AND (NOT HELP) (OR (GETF INITARGS :CLEAN) (GETF INITARGS :DEEP-CLEAN))) (FORMAT T "Cleaning up previous build~%~%") (UIOP/FILESYSTEM:DELETE-DIRECTORY-TREE BUILD :VALIDATE T :IF-DOES-NOT-EXIST :IGNORE)) (UNLESS HELP (FORMAT T "Synchronizing external repositories~%~%")) (LOOP WITH DEEP-CLEAN = (GETF INITARGS :DEEP-CLEAN) FOR SOURCE IN (UIOP/STREAM:READ-FILE-FORM #P"repos.sexp") FOR NAME = ...) (ASDF/SOURCE-REGISTRY:INITIALIZE-SOURCE-REGISTRY (QUASIQUOTE (:SOURCE-REGISTRY (:ALSO-EXCLUDE "asdf") (:TREE #) :INHERIT-CONFIGURATION))) (ASDF/OPERATE:LOAD-SYSTEM :KOGA) (APPLY (FUNCTION UIOP/PACKAGE:SYMBOL-CALL) "KOGA" (IF HELP "HELP" "SETUP") INITARGS)) #<NULL-LEXENV>)
16: (EVAL-TLF (LET* ((INITARGS (NCONC (PARSE-COMMAND-LINE-ARGUMENTS) (IGNORE-ERRORS #) (IGNORE-ERRORS #))) (*GIT* (GETF INITARGS :GIT "git")) (BUILD (UIOP/PATHNAME:ENSURE-DIRECTORY-PATHNAME (GETF INITARGS :BUILD-PATH "build/"))) (EXTENSIONS (GETF INITARGS :EXTENSIONS)) (SKIP-SYNC (GETF INITARGS :SKIP-SYNC)) (HELP (GETF INITARGS :HELP))) (WHEN (AND (NOT HELP) (OR (GETF INITARGS :CLEAN) (GETF INITARGS :DEEP-CLEAN))) (FORMAT T "Cleaning up previous build~%~%") (UIOP/FILESYSTEM:DELETE-DIRECTORY-TREE BUILD :VALIDATE T :IF-DOES-NOT-EXIST :IGNORE)) (UNLESS HELP (FORMAT T "Synchronizing external repositories~%~%")) (LOOP WITH DEEP-CLEAN = (GETF INITARGS :DEEP-CLEAN) FOR SOURCE IN (UIOP/STREAM:READ-FILE-FORM #P"repos.sexp") FOR NAME = ...) (ASDF/SOURCE-REGISTRY:INITIALIZE-SOURCE-REGISTRY (QUASIQUOTE (:SOURCE-REGISTRY (:ALSO-EXCLUDE "asdf") (:TREE #) :INHERIT-CONFIGURATION))) (ASDF/OPERATE:LOAD-SYSTEM :KOGA) (APPLY (FUNCTION UIOP/PACKAGE:SYMBOL-CALL) "KOGA" (IF HELP "HELP" "SETUP") INITARGS)) 11 NIL)
17: ((LABELS SB-FASL::EVAL-FORM :IN LOAD-AS-SOURCE) (LET* ((INITARGS (NCONC (PARSE-COMMAND-LINE-ARGUMENTS) (IGNORE-ERRORS #) (IGNORE-ERRORS #))) (*GIT* (GETF INITARGS :GIT "git")) (BUILD (UIOP/PATHNAME:ENSURE-DIRECTORY-PATHNAME (GETF INITARGS :BUILD-PATH "build/"))) (EXTENSIONS (GETF INITARGS :EXTENSIONS)) (SKIP-SYNC (GETF INITARGS :SKIP-SYNC)) (HELP (GETF INITARGS :HELP))) (WHEN (AND (NOT HELP) (OR (GETF INITARGS :CLEAN) (GETF INITARGS :DEEP-CLEAN))) (FORMAT T "Cleaning up previous build~%~%") (UIOP/FILESYSTEM:DELETE-DIRECTORY-TREE BUILD :VALIDATE T :IF-DOES-NOT-EXIST :IGNORE)) (UNLESS HELP (FORMAT T "Synchronizing external repositories~%~%")) (LOOP WITH DEEP-CLEAN = (GETF INITARGS :DEEP-CLEAN) FOR SOURCE IN (UIOP/STREAM:READ-FILE-FORM #P"repos.sexp") FOR NAME = ...) (ASDF/SOURCE-REGISTRY:INITIALIZE-SOURCE-REGISTRY (QUASIQUOTE (:SOURCE-REGISTRY (:ALSO-EXCLUDE "asdf") (:TREE #) :INHERIT-CONFIGURATION))) (ASDF/OPERATE:LOAD-SYSTEM :KOGA) (APPLY (FUNCTION UIOP/PACKAGE:SYMBOL-CALL) "KOGA" (IF HELP "HELP" "SETUP") INITARGS)) 11)
18: ((LAMBDA (FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN LOAD-AS-SOURCE) (LET* ((INITARGS (NCONC (PARSE-COMMAND-LINE-ARGUMENTS) (IGNORE-ERRORS #) (IGNORE-ERRORS #))) (*GIT* (GETF INITARGS :GIT "git")) (BUILD (UIOP/PATHNAME:ENSURE-DIRECTORY-PATHNAME (GETF INITARGS :BUILD-PATH "build/"))) (EXTENSIONS (GETF INITARGS :EXTENSIONS)) (SKIP-SYNC (GETF INITARGS :SKIP-SYNC)) (HELP (GETF INITARGS :HELP))) (WHEN (AND (NOT HELP) (OR (GETF INITARGS :CLEAN) (GETF INITARGS :DEEP-CLEAN))) (FORMAT T "Cleaning up previous build~%~%") (UIOP/FILESYSTEM:DELETE-DIRECTORY-TREE BUILD :VALIDATE T :IF-DOES-NOT-EXIST :IGNORE)) (UNLESS HELP (FORMAT T "Synchronizing external repositories~%~%")) (LOOP WITH DEEP-CLEAN = (GETF INITARGS :DEEP-CLEAN) FOR SOURCE IN (UIOP/STREAM:READ-FILE-FORM #P"repos.sexp") FOR NAME = ...) (ASDF/SOURCE-REGISTRY:INITIALIZE-SOURCE-REGISTRY (QUASIQUOTE (:SOURCE-REGISTRY (:ALSO-EXCLUDE "asdf") (:TREE #) :INHERIT-CONFIGURATION))) (ASDF/OPERATE:LOAD-SYSTEM :KOGA) (APPLY (FUNCTION UIOP/PACKAGE:SYMBOL-CALL) "KOGA" (IF HELP "HELP" "SETUP") INITARGS)) :CURRENT-INDEX 11)
19: (SB-C::%DO-FORMS-FROM-INFO #<FUNCTION (LAMBDA (FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN LOAD-AS-SOURCE) {70087E03EB}> #<SB-C::SOURCE-INFO {70087E0423}> SB-C::INPUT-ERROR-IN-LOAD)
20: (LOAD-AS-SOURCE #<FD-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/./koga" {70087E0003}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
21: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<FD-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/./koga" {70087E0003}> NIL)
22: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {1010D09EB}> #<FD-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/./koga" {70087E0003}> NIL #<FD-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/./koga" {70087E0003}>)
23: (LOAD #<FD-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/./koga" {70087E0003}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT)
24: ((FLET LOAD-SCRIPT :IN PROCESS-SCRIPT) #<FD-STREAM for "file /Users/rjsheperd/Code/rjsheperd/clasp/./koga" {70087E0003}>)
25: ((FLET SB-UNIX::BODY :IN PROCESS-SCRIPT))
26: ((FLET "WITHOUT-INTERRUPTS-BODY-11" :IN PROCESS-SCRIPT))
27: (PROCESS-SCRIPT "./koga")
28: (TOPLEVEL-INIT)
29: ((FLET SB-UNIX::BODY :IN START-LISP))
30: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN START-LISP))
31: (%START-LISP)

unhandled condition in --disable-debugger mode, quitting

EDIT For clarity, here is my OS/Hardware info:

Software:

      System Version: macOS 14.0 (23A344)
      Kernel Version: Darwin 23.0.0

Hardware:

    Hardware Overview:

      Model Name: MacBook Air
      Model Identifier: MacBookAir10,1
      Model Number: MGN73LL/A
      Chip: Apple M1
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 8 GB
      System Firmware Version: 10151.1.1
      OS Loader Version: 10151.1.1

rjsheperd avatar Oct 24 '23 18:10 rjsheperd

No. It won't build because the JIT code on ARM doesn't work yet. Not sure why you having an error earlier in the build, though.

yitzchak avatar Oct 24 '23 19:10 yitzchak

It would be great if native aarch64 would be supported by Clasp. More and more Mac users are on this platform.

mdbergmann avatar Dec 11 '23 18:12 mdbergmann

Hello there, just checking in here to see if there's any updates on this 🙂

cjbarre avatar May 04 '24 01:05 cjbarre

Not yet. Worked on it a bit last week. I'll update the thread when I have new info.

yitzchak avatar May 04 '24 12:05 yitzchak

Not yet. Worked on it a bit last week. I'll update the thread when I have new info. Thanks @yitzchak I see that CLASP_APPLE_SILICON macro defined! FWIW with llvm@18 on macOS arm64 building stage 1 and 2 build fine. So that definitely progress 🥳 But boehmprecise/iclasp fails to compile both ../src/lisp/modules/serve-event/serve-event.lisp and ../src/lisp/modules/asdf/build/asdf.lisp:

λ ninja -C build
ninja: Entering directory `build'
[1/2] Compiling module ../src/lisp/modules/serve-event/serve-event.lisp
FAILED: boehmprecise/lib/modules/serve-event.faso
boehmprecise/iclasp --non-interactive --norc --disable-mpi --base --feature ignore-extensions --load compile-module.lisp -- "SYS:LIB;MODULES;SERVE-EVENT.FASO" "SYS:SRC;LISP;MODULES;SERVE-EVENT;SERVE-EVENT.LISP"
Starting clasp-boehmprecise-2.5.0-243-g95198e563 from base image
datum: CORE::WRONG-NUMBER-OF-ARGUMENTS (:CALLED-FUNCTION #<$CORE::CLOSURE @0x1065fa2d9$> :GIVEN-NARGS 3 :MIN-NARGS 4 :MAX-NARGS 4)


universalErrorHandler
../src/core/foundation.cc:341 dbg_hook(...) was called
[2/2] Compiling module ../src/lisp/modules/asdf/build/asdf.lisp
FAILED: boehmprecise/lib/modules/asdf.faso
boehmprecise/iclasp --non-interactive --norc --disable-mpi --base --feature ignore-extensions --load compile-module.lisp -- "SYS:LIB;MODULES;ASDF.FASO" "SYS:SRC;LISP;MODULES;ASDF;BUILD;ASDF.LISP"
Starting clasp-boehmprecise-2.5.0-243-g95198e563 from base image
datum: COMMON-LISP::TYPE-ERROR (:DATUM #(13 2 15 14 13 12 3 6 5 0 12 1 0 0 0 0 0 )@0x104d3fd19 :EXPECTED-TYPE CORE::SIMPLE-VECTOR-BYTE8-T)


universalErrorHandler
../src/core/foundation.cc:341 dbg_hook(...) was called

so wrong number of args in one case, wrong argument type in the other.

jsulmont avatar May 18 '24 11:05 jsulmont

@jsulmont I'm aware of this. They are indicators that something is wrong with the stamps of literals loaded from a FASO.

yitzchak avatar May 18 '24 14:05 yitzchak

@jsulmont I'm aware of this. They are indicators that something is wrong with the stamps of literals loaded from a FASO.

Thanks @yitzchak 👍🏻

jsulmont avatar May 19 '24 09:05 jsulmont

@yitzchak perhaps if you point me at the right direction I can spend some time on this and see if I am able to help.

jsulmont avatar May 25 '24 07:05 jsulmont