Python 3.10.*, 3.9.* Install fails on Intel Mac OS X 11.6.2 and 12.3.1 with linker error with symbols not found for x86_64

[richtong]

ugh, so many Mac-related build problems, this homebrew installs fine for 3.10.1 and 3.9.9.

python-build 3.10.1 /Users/rich/.asdf/installs/python/3.10.1
python-build: use [email protected] from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.1.tar.xz...
-> https://www.python.org/ftp/python/3.10.1/Python-3.10.1.tar.xz
Installing Python-3.10.1...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.6.2 using python-build 2.2.3-1-g423de9ae)

Inspect or clean up the working tree at /var/folders/2q/m0b64f1n73xb3rpz1gxfd6m00000gq/T/python-build.20220104201327.37355

Results logged to /var/folders/2q/m0b64f1n73xb3rpz1gxfd6m00000gq/T/python-build.20220104201327.37355.log

Last 10 log lines:
  "__Py_Dealloc", referenced from:
      _test_open_code_hook in _testembed.o
      _test_unicode_id_init in _testembed.o
      __audit_hook_run in _testembed.o
  "__Py_InitializeMain", referenced from:
      _test_init_main in _testembed.o
      _test_init_set_config in _testembed.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:736: Programs/_testembed] Error 1

In looking at the Log files I see this line which is looking for a library file that is in ~/.asdf/install/python but this directory is empty so it seems like that's the problem

ld: warning: ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.10.0/lib'
   5 directory not found for option '-L/Users/rich/.asdf/installs/python/3.10.0/lib'
   4 ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.10.0/lib'
   3 ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.10.0/lib'
   2 ld: warning: ignoring file libpython3.10.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
   1 ld: warning: ignoring file libpython3.10.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
940  Undefined symbols for architecture x86_64:

[amandafanny]

I have same problem

[richtong]

OK, this also happens on MacOS 12.2.1 with python 3.10.0, 3.9.10

install-asdf.sh: asdf install python 3.9.10
python-build 3.9.10 /Users/rich/.asdf/installs/python/3.9.10
python-build: use [email protected] from homebrew
python-build: use readline from homebrew
Downloading Python-3.9.10.tar.xz...
-> https://www.python.org/ftp/python/3.9.10/Python-3.9.10.tar.xz
Installing Python-3.9.10...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 12.2.1 using python-build 2.2.4-1-4-g1e79a522)

Inspect or clean up the working tree at /var/folders/mv/882fw5dx12d0kntz9rxdz78r0000gq/T/python-build.20220219150100.16430
Results logged to /var/folders/mv/882fw5dx12d0kntz9rxdz78r0000gq/T/python-build.20220219150100.16430.log

Last 10 log lines:
  "__PyRuntime", referenced from:
      _check_preinit_isolated_config in _testembed.o
  "__Py_Dealloc", referenced from:
      _test_open_code_hook in _testembed.o
      __audit_hook_run in _testembed.o
  "__Py_InitializeMain", referenced from:
      _test_init_main in _testembed.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:739: Programs/_testembed] Error 1

Looking at the log files I see clang compilation step is failing first missing ~/.asdf/installs/python which I can confirm is completely empty and the libpython3.9.a

clang -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/rich/.asdf/installs/python/3.9.6/lib  -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/rich/.asdf/installs/python/3.9.6/lib  -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib   -Wl,-stack_size,1000000  -framework CoreFoundation -o Programs/_testembed Programs/_testembed.o libpython3.9.a -lintl -ldl   -framework CoreFoundation
  29 ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.9.6/lib'
  28 ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.9.6/lib'
  27 ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.9.6/lib'
  26 ld: warning: directory not found for option '-L/Users/rich/.asdf/installs/python/3.9.6/lib'
  25 ld: warning: ld: warning: ignoring file libpython3.9.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
  24 ignoring file libpython3.9.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )

[richtong]

And for Monterey as well, I'm thinking I need to blow away my ASDF installation and see what happens. This is working fine on one of my Intel Macs but not the other. It seems to work on my M1 Mac, so could be installation related. Sigh.

[richtong]

I tried a complete delete. Removing asdf and ~/.asdf and this still happens sigh. However I have another MacBook Pro Intel 2016 which does not have this problem. Things correctly install. Any ideas on how to do a clean reinstall?

[tomwganem]

Having some issue as well

[tomwganem]

But I also have the same issue for ruby, erlang, and elixir.

[lkollar]

Anyone else running into this issue, make sure the binutils package path does not come before the system tools, as otherwise those could be picked up during build and they can produce incompatible output. In particular, which ranlib should report /usr/bin/ranlib.

[richtong]

Got it trying that now, so it needs the latest binutils, interesting this isn't a problem on M1

[richtong]

I'm having trouble figuring out how to get the right ranlib as I'm using homebrew and just putting /usr/bin in front of the PATH causes all kinds of other problems. Sigh.

[offwork]

I don't think this issue will be resolved for now. The issue depends on many other issues, for example the c/c++ compiler mac os uses or the default python version on mac and even brew's dependencies.

[richtong]

Ok I'm trying a clean install. Very confusing.