Workaround for passing givaro library to linker on OSX

[vbraun]

Just hack the missing -lgivaro into the linker command, see also https://github.com/linbox-team/fflas-ffpack/issues/391 for the real bug

Closes https://github.com/sagemath/sage/issues/38002

[mkoeppe]

This works well, thanks

[vbraun]

Linbox then fails to build on macos because const_cast<size_t&>(ld) should have been const_cast<size_t&>(_ld), I guess this is elided in gcc 14 so the error is not detected there. I've adapted the linbox patch.

[github-actions[bot]]

Documentation preview for this PR (built with commit 8354962100dd793808898171499269f16f56655b; changes) is ready! :tada: This preview will update shortly after each push to this PR.

[mkoeppe]

Can confirm that this fixes this error.

I'm then running into

[linbox-1.7.0] [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
[linbox-1.7.0] [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)

this could be a misconfiguration related to gfortran though, I'll try again in a clean worktree

[mkoeppe]

That was it. linbox builds now.

[kwankyu]

Can confirm that this fixes this error.

I'm then running into

[linbox-1.7.0] [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
[linbox-1.7.0] [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)

I am getting the same error.

this could be a misconfiguration related to gfortran though, I'll try again in a clean worktree

Do you mean something else than make distclean?

[mkoeppe]

Just make distclean

[kwankyu]

I did it, but still get the error...

[kwankyu]

For me,

Checking whether SageMath should install SPKG gfortran...
checking the version of gfortran... 14
configure: will use system package and not install SPKG gfortran

[mkoeppe]

Here I'm using this:

gfortran --version
GNU Fortran (Homebrew GCC 13.2.0) 13.2.0

I've started tests at https://github.com/mkoeppe/sage/actions/runs/9136322984

[kwankyu]

Here

$ gfortran --version
GNU Fortran (Homebrew GCC 14.1.0) 14.1.0

I am installing gfortran spkg (gfortran 12.2) to see if this helps...

[mkoeppe]

that will probably not build; you'll need #38008

[kwankyu]

It worked:

...
Finished installing linbox-1.7.0
1m2.13s real 42.99s user 16.48s sys  

[mkoeppe]

Error reproduced in local-macos (13, xcode_15.0, homebrew-macos-usrlocal-standard)](https://github.com/mkoeppe/sage/actions/runs/9136322984/job/25124971687#logs)

 [linbox-1.7.0]   [spkg-install] libtool: link: g++ -std=gnu++11 -std=gnu++11 -dynamiclib -Wl,-flat_namespace -Wl,-undefined -Wl,suppress -o .libs/liblinbox.0.dylib .libs/liblinbox.0.dylib-master.o  -L/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib -lntl -lmpfr -liml -lflint  -O2 -march=native -pthread -g -O2 -Wl,-rpath -Wl,/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib -Wl,-ld_classic -Wl,-rpath -Wl,/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib   -pthread -install_name  /Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib/liblinbox.0.dylib -compatibility_version 1 -current_version 1.0
  [linbox-1.7.0]   [spkg-install] ld: warning: -undefined suppress is deprecated
  [linbox-1.7.0]   [spkg-install] ld: warning: duplicate -rpath '/Users/runner/work/sage/sage/.tox/local-homebrew-macos-usrlocal-standard-xcode_15.0/local/lib' ignored
  [linbox-1.7.0]   [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
  [linbox-1.7.0]   [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)
  [linbox-1.7.0]   [spkg-install] make[6]: *** [liblinbox.la] Error 1

Likewise local-macos (latest, homebrew-macos-opthomebrew-standard)

[kwankyu]

... but eventually failed to build sage:

[sagelib-10.4.beta6] [spkg-install]     gcc -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -g -O2 build/temp.macosx-12.0-x86_64-cpython-312/sage/schemes/elliptic_curves/period_lattice_region.o -o build/lib.macosx-12.0-x86_64-cpython-312/sage/schemes/elliptic_curves/period_lattice_region.cpython-312-darwin.so
[sagelib-10.4.beta6] [spkg-install]     ld: warning: -undefined dynamic_lookup may not work with chained fixups
[sagelib-10.4.beta6] [spkg-install]     24 warnings generated.
[sagelib-10.4.beta6] [spkg-install]     gcc -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -L/Users/kwankyu/GitHub/sage-dev/local/lib -Wl,-rpath,/Users/kwankyu/GitHub/sage-dev/local/lib -g -O2 build/temp.macosx-12.0-x86_64-cpython-312/sage/stats/time_series.o -o build/lib.macosx-12.0-x86_64-cpython-312/sage/stats/time_series.cpython-312-darwin.so -lpari
[sagelib-10.4.beta6] [spkg-install]     ld: warning: -undefined dynamic_lookup may not work with chained fixups
[sagelib-10.4.beta6] [spkg-install]     error: command '/usr/bin/gcc' failed with exit code 1
[sagelib-10.4.beta6] [spkg-install]     error: subprocess-exited-with-error
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     × python setup.py develop did not run successfully.
[sagelib-10.4.beta6] [spkg-install]     │ exit code: 1
[sagelib-10.4.beta6] [spkg-install]     ╰─> See above for output.
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     note: This error originates from a subprocess, and is likely not a problem with pip.
[sagelib-10.4.beta6] [spkg-install]     full command: /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.12/bin/python3 -c '
[sagelib-10.4.beta6] [spkg-install]     exec(compile('"'"''"'"''"'"'
[sagelib-10.4.beta6] [spkg-install]     # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
[sagelib-10.4.beta6] [spkg-install]     #
[sagelib-10.4.beta6] [spkg-install]     # - It imports setuptools before invoking setup.py, to enable projects that directly
[sagelib-10.4.beta6] [spkg-install]     #   import from `distutils.core` to work with newer packaging standards.
[sagelib-10.4.beta6] [spkg-install]     # - It provides a clear error message when setuptools is not installed.
[sagelib-10.4.beta6] [spkg-install]     # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
[sagelib-10.4.beta6] [spkg-install]     #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
[sagelib-10.4.beta6] [spkg-install]     #     manifest_maker: standard file '"'"'-c'"'"' not found".
[sagelib-10.4.beta6] [spkg-install]     # - It generates a shim setup.py, for handling setup.cfg-only projects.
[sagelib-10.4.beta6] [spkg-install]     import os, sys, tokenize
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     try:
[sagelib-10.4.beta6] [spkg-install]         import setuptools
[sagelib-10.4.beta6] [spkg-install]     except ImportError as error:
[sagelib-10.4.beta6] [spkg-install]         print(
[sagelib-10.4.beta6] [spkg-install]             "ERROR: Can not execute `setup.py` since setuptools is not available in "
[sagelib-10.4.beta6] [spkg-install]             "the build environment.",
[sagelib-10.4.beta6] [spkg-install]             file=sys.stderr,
[sagelib-10.4.beta6] [spkg-install]         )
[sagelib-10.4.beta6] [spkg-install]         sys.exit(1)
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     __file__ = %r
[sagelib-10.4.beta6] [spkg-install]     sys.argv[0] = __file__
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     if os.path.exists(__file__):
[sagelib-10.4.beta6] [spkg-install]         filename = __file__
[sagelib-10.4.beta6] [spkg-install]         with tokenize.open(__file__) as f:
[sagelib-10.4.beta6] [spkg-install]             setup_py_code = f.read()
[sagelib-10.4.beta6] [spkg-install]     else:
[sagelib-10.4.beta6] [spkg-install]         filename = "<auto-generated setuptools caller>"
[sagelib-10.4.beta6] [spkg-install]         setup_py_code = "from setuptools import setup; setup()"
[sagelib-10.4.beta6] [spkg-install]     
[sagelib-10.4.beta6] [spkg-install]     exec(compile(setup_py_code, filename, "exec"))
[sagelib-10.4.beta6] [spkg-install]     '"'"''"'"''"'"' % ('"'"'/Users/kwankyu/GitHub/sage-dev/src/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' --no-user-cfg develop --no-deps
[sagelib-10.4.beta6] [spkg-install]     cwd: /Users/kwankyu/GitHub/sage-dev/src/
[sagelib-10.4.beta6] [spkg-install] error: subprocess-exited-with-error
[sagelib-10.4.beta6] [spkg-install] 
[sagelib-10.4.beta6] [spkg-install] × python setup.py develop did not run successfully.
[sagelib-10.4.beta6] [spkg-install] │ exit code: 1
[sagelib-10.4.beta6] [spkg-install] ╰─> See above for output.
[sagelib-10.4.beta6] [spkg-install] 
[sagelib-10.4.beta6] [spkg-install] note: This error originates from a subprocess, and is likely not a problem with pip.
[sagelib-10.4.beta6] [spkg-install] ********************************************************************************
[sagelib-10.4.beta6] [spkg-install] Error installing sagelib-10.4.beta6
[sagelib-10.4.beta6] [spkg-install] ********************************************************************************
[sagelib-10.4.beta6] [spkg-install] 1m17.87s real 3m3.21s user 19.39s sys
[sagelib-10.4.beta6] ************************************************************************
[sagelib-10.4.beta6] Error installing package sagelib-10.4.beta6

I don't know if this is related though...

[mkoeppe]

The real error is likely much further up in the sagelib*.log

[kwankyu]

Right. There are lots of errors like:

[sagelib-10.4.beta6] [spkg-install]     /usr/local/include/flint/padic.h:22:10: fatal error: 'padic_types.h' file not found
[sagelib-10.4.beta6] [spkg-install]     #include "padic_types.h"
[sagelib-10.4.beta6] [spkg-install]              ^~~~~~~~~~~~~~~
[sagelib-10.4.beta6] [spkg-install]     4 warnings and 1 error generated.

[kwankyu]

Building again from scratch...with this PR.

[mkoeppe]

There are lots of errors like:

[sagelib-10.4.beta6] [spkg-install]     /usr/local/include/flint/padic.h:22:10: fatal error: 'padic_types.h' file not found
[sagelib-10.4.beta6] [spkg-install]     #include "padic_types.h"
[sagelib-10.4.beta6] [spkg-install]              ^~~~~~~~~~~~~~~
[sagelib-10.4.beta6] [spkg-install]     4 warnings and 1 error generated.

I also see this one in CI: stage-2 / local-macos (12, homebrew-macos-usrlocal-standard)

[kwankyu]

Building again from scratch...with this PR.

Then it stopped at the gfortran problem. So I did make gfortran to install gfortran 12.2 to avoid using the system gfortran 14.1

Then it stopped at "padic_types.h" problem. I did make flint. I guess that somehow sagelib had started to build before flint was built.

Then it is now building the doc. So the building seems to be going to finish well.

[mkoeppe]

Homebrew's flint package (both version 3.0.1 and 3.1.0) appears to be broken

$ ls /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h      
/usr/local/opt/flint/include/flint/padic_mat.h  
/usr/local/opt/flint/include/flint/padic_poly.h
$ grep padic_typ /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_mat.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_poly.h:#include "padic_types.h"

[mkoeppe]

I'm then running into

[linbox-1.7.0] [spkg-install] ld: file not found: @rpath/libquadmath.0.dylib for architecture x86_64
[linbox-1.7.0] [spkg-install] clang: error: linker command failed with exit code 1 (use -v to see invocation)

Also this one can be fixed by running autoreconf on the linbox source tree, same as https://github.com/sagemath/sage/issues/38002#issuecomment-2118295596

[mkoeppe]

I regenerated fflas-ffpack release tarballs at https://github.com/mkoeppe/fflas-ffpack/releases using ubuntu-jammy and ubuntu-24.04. But these regenerated tarballs do not work either -- because Debian patches libtool to death: https://sources.debian.org/patches/libtool/2.4.7-7/link_all_deplibs.patch/

[mkoeppe]

This one, https://github.com/mkoeppe/fflas-ffpack/releases/tag/v2.5.0%2Bsage-2024-05-18b, regenerated using upstream GNU libtool 2.4.7, works.

[mkoeppe]

Here's the release for linbox: https://github.com/mkoeppe/linbox/releases/tag/v1.7.0%2Bsage-2024-05-18

[mkoeppe]

Follow-up PR that uses these two fixed tarballs: #38025.

[dcoudert]

Building again from scratch...with this PR.

Then it stopped at the gfortran problem. So I did make gfortran to install gfortran 12.2 to avoid using the system gfortran 14.1

Then it stopped at "padic_types.h" problem. I did make flint. I guess that somehow sagelib had started to build before flint was built.

Then it is now building the doc. So the building seems to be going to finish well.

Almost the same for me with this PR: make build failed, then make gfortran, then make build -j4 failed, then make build succeed. I have not tried without this PR.

[kwankyu]

Homebrew's flint package (both version 3.0.1 and 3.1.0) appears to be broken

$ ls /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h      
/usr/local/opt/flint/include/flint/padic_mat.h  
/usr/local/opt/flint/include/flint/padic_poly.h
$ grep padic_typ /usr/local/opt/flint/include/flint/padic*  
/usr/local/opt/flint/include/flint/padic.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_mat.h:#include "padic_types.h"
/usr/local/opt/flint/include/flint/padic_poly.h:#include "padic_types.h"

Ah, I see. Thanks.