unbound
unbound copied to clipboard
NLnetLabs
Cannot compile unbound with pyunbound on mac
I'm trying to compile unbound on a mac, i specifically need the python bindings for internet.nl development. To get these bindings the developer headers from python are needed. I've used a dozen different directories for the header files, from both Mac OS and Homebrew. None of them seem to satisfy the requirements from configure, even if they contain python.h or the proper dylibs.
For example:
./configure LDFLAGS="-L/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers" --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis
Other pats tried are:
-
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers -
/opt/homebrew/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/Current/include/python3.8/ -
/opt/homebrew/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/include/python3.8/
... and any that contain lib
Expected behavior Compilation of unbound or any more specific message what is wrong.
System:
- Unbound version: https://github.com/internetstandards/unbound
- OS: Mac OS 11.6 (Big Sur)
-
unbound -Voutput: -
Additional information Add any other information that you may have gathered about the issue here.
The python detection code uses the variable PYTHON_VERSION that can be passed to detect python directories. Try setting PYTHON_VERSION=3.8 with like ./configure --with-pyunbound PYTHON_VERSION=3.8
The configure script then uses that version of python distutils with python3.8 and gets the directories needed.
If that does not work you can influence the results that are gathered from the python3.8 version, by setting the variables: PYTHON (to set the python commandline executable to use), and the directories used for compilation individually with PYTHON_CPPFLAGS and PYTHON_LDFLAGS and PYTHON_SITE_PKG to the directories that you want to use. But the easy method is to set PYTHON_VERSION and have it detect the settings from there.
Thanks for the quick response!
I see, switching the python version around does seem to discover alternative folders. Those seem to be correct-ish, while none of them contains Python.h. Setting PYTHON_LDFLAGS to a directory that does contain these file still does not has any impact on the outcome.
Switching around different python versions results in paths that seem fine. But still the compilation fails.
3.7:
checking for python3.7... /usr/local/bin/python3.7
checking for the distutils Python package... yes
checking for Python include path... -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m
checking for Python library path... -L/Library/Frameworks/Python.framework/Versions/3.7/lib -L/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
checking for Python site-packages path... /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages
3.8:
checking for python3.8... /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
checking for the distutils Python package... yes
checking for Python include path... -I/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8
checking for Python library path... -L/Library/Frameworks/Python.framework/Versions/3.8/lib -L/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8
checking for Python site-packages path... /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
3.9:
checking for python3.9... /opt/homebrew/bin/python3.9
checking for the distutils Python package... yes
checking for Python include path... -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9
checking for Python library path... -L/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/lib -L/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/lib/python3.9
checking for Python site-packages path... /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages
Catting conftest.err results in:
undef: _Py_Initialize
Undefined symbols for architecture arm64:
"_Py_Initialize", referenced from:
_main in cc-dcb158.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
no
So the problem is not with the paths, but with the x86 / arm64 issue. I'm trying to compile it on an arm64 macbook (m1). Perhaps that usecase is not supported?
Seems to be a deadlock:
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis
results in: ld: symbol(s) not found for architecture arm64
arch -x86_64 ./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis
results in ld: symbol(s) not found for architecture x86_64
Unbound and I guess python can compile for arm64 just fine, we have unit tests that compile on arm64. Perhaps the homebrew installed binaries are for the wrong architecture, x86 instead of arm64?
The library path that it has found contains the python library? And it tries to link that in the config.log output? If so I guess you are right that it is the arm64.
What is it exactly trying to do that gives the symbol not found error? config.log says what it does.
I installed [email protected] from brew and followed the instructions. The paths are:
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhredis PYTHON_LDFLAGS=/opt/homebrew/opt/[email protected]/lib PYTHON_CPPFLAGS=/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8
Config.log says:
conftest.c:85:26: fatal error: 'Python.h' file not found #include <Python.h> ^~~~~~~~~~ 1 error generated. configure:17396: $? = 1 configure: failed program was:
The contents of ls /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8 includes Python.h and many other .h files... specifically for version 3.8 of python as they come with the brew package. I've also tried several completely new other python installs.
The permissions of Python.h:
-rw-r--r-- 1 someuser somegroup 3615 Aug 30 18:42 Python.h
Gcc has root permissions.
The content of Python.h looks fine.
Running the compliation command as root still results in Python.h file not found.
The gcc version i'm using:
(.venv) macbookprom1:unbound stitch$ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1 Apple clang version 13.0.0 (clang-1300.0.29.3) Target: arm64-apple-darwin20.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
I've tried it with the homebrew version of gcc:
/opt/homebrew/Cellar/gcc/11.2.0_1/bin/gcc-11 -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8 conftest.c /opt/homebrew/opt/[email protected]/lib conftest.c:1:10: fatal error: Python.h: No such file or directory 1 | #include <Python.h>
Here is the more verbose config.log output:
configure:17251: checking for python
configure:17269: found /Users/stitch/Documents/_webdevelopment/internetnl/Internet.nl/.venv/bin/python
configure:17281: result: /Users/stitch/Documents/_webdevelopment/internetnl/Internet.nl/.venv/bin/python
configure:17302: checking for the distutils Python package
configure:17305: result: yes
configure:17319: checking for Python include path
configure:17329: result: /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8
configure:17336: checking for Python library path
configure:17342: result: /opt/homebrew/opt/[email protected]/lib
configure:17354: checking for Python site-packages path
configure:17360: result: /Users/stitch/Documents/_webdevelopment/internetnl/Internet.nl/.venv/lib/python3.8/site-packages
configure:17367: checking consistency of all components of python development environment
configure:17396: gcc -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8 conftest.c /opt/homebrew/opt/[email protected]/lib >&5
conftest.c:85:26: fatal error: 'Python.h' file not found
#include <Python.h>
^~~~~~~~~~
1 error generated.
configure:17396: $? = 1
My compiler skills are lacking clearly. All my assumptions of including files based on a path are now over board. Where am i going wrong?
For reference:
ls /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8
Python-ast.h cellobject.h dictobject.h genobject.h marshal.h osdefs.h pydtrace.h pyport.h structmember.h warnings.h
Python.h ceval.h dtoa.h graminit.h memoryobject.h osmodule.h pydtrace_probes.h pystate.h structseq.h weakrefobject.h
abstract.h classobject.h dynamic_annotations.h grammar.h methodobject.h parsetok.h pyerrors.h pystrcmp.h symtable.h
asdl.h code.h enumobject.h import.h modsupport.h patchlevel.h pyexpat.h pystrhex.h sysmodule.h
ast.h codecs.h errcode.h internal moduleobject.h picklebufobject.h pyfpe.h pystrtod.h token.h
bitset.h compile.h eval.h interpreteridobject.h namespaceobject.h py_curses.h pyhash.h pythonrun.h traceback.h
bltinmodule.h complexobject.h fileobject.h intrcheck.h node.h pyarena.h pylifecycle.h pythread.h tracemalloc.h
boolobject.h context.h fileutils.h iterobject.h object.h pycapsule.h pymacconfig.h pytime.h tupleobject.h
bytearrayobject.h cpython floatobject.h listobject.h objimpl.h pyconfig.h pymacro.h rangeobject.h typeslots.h
bytes_methods.h datetime.h frameobject.h longintrepr.h odictobject.h pyctype.h pymath.h setobject.h ucnhash.h
bytesobject.h descrobject.h funcobject.h longobject.h opcode.h pydebug.h pymem.h sliceobject.h unicodeobject.h
You are missing the -L and -I flags that the LDFLAGS and CPPFLAGS need in front of the directory. That is needed to specify how the compile treats the directory, for searching libraries or searching includes.
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhredis PYTHON_LDFLAGS=-L/opt/homebrew/opt/[email protected]/lib PYTHON_CPPFLAGS=-I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8
Thanks for the quick response! :)
Ah, that's how it's picked up. Still being squished between non existing symbols.
arm64 (default):
gcc -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8 manualtest.c -L/opt/homebrew/opt/[email protected]/lib
undef: _Py_Initialize
Undefined symbols for architecture arm64:
"_Py_Initialize", referenced from:
_main in cc-6e745a.o
ld: symbol(s) not found for architecture arm64
x86_64:
arch -x86_64 gcc -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8 manualtest.c -L/opt/homebrew/opt/[email protected]/lib
undef: _Py_Initialize
Undefined symbols for architecture x86_64:
"_Py_Initialize", referenced from:
_main in cc-cccdce.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The python from brew is a 64 bit one:
file /opt/homebrew/opt/[email protected]/bin/python3
/opt/homebrew/opt/[email protected]/bin/python3: Mach-O 64-bit executable arm64
That could be because -lpython3.8 is missing from the commandline, because you override the python checks with your own values. Try adding -lpython3.8 to the PYTHON_LDFLAGS:
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhredis "PYTHON_LDFLAGS=-L/opt/homebrew/opt/[email protected]/lib -lpython3.8" PYTHON_CPPFLAGS=-I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/include/python3.8
Should 'lpython3.8' be the python binary?
In that case, no luck pointing to it directly and indirectly.
ld: library not found for -lpython3.8
Yes it is the python binary, the libpython file. Without the 'lib' part, or the '.so' or '.a' file extension. The file should be in the lib directory.
What's the content of a normal 'lib' directory?
The instructions from brew point to this:
ls /opt/homebrew/opt/[email protected]/lib
pkgconfig
Doesn't seem right :)
You need a file like libpython3.8.a or libpython3.8.so or something like that. Pass the directory where that file is, perhaps in lib64, if a directory with that name exists?
Still no luck, it seems like an arm64 version of python is being mixed with the x86_64 one, even after making all python variables explicitly use the x86_64 one. I explicitly need an x86_64 version of the library, given there are some dependencies in the clients repo that will probably not support arm64.
1: get the x86_64 version of brew to install python3.9:
arch -x86_64 /bin/bash
/usr/local/Homebrew/bin/brew install [email protected]
2: Verify that the this cannot be run on arm64:
arch -arm64 /usr/local/Cellar/[email protected]/3.9.9/bin/python3.9
arch: posix_spawnp: /usr/local/Cellar/[email protected]/3.9.9/bin/python3.9: Bad CPU type in executable
bash-3.2$ arch -x86_64 /usr/local/Cellar/[email protected]/3.9.9/bin/python3.9
Python 3.9.9 (main, Nov 21 2021, 03:23:44)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
3: Check the contents of the include path (-I):
ls /usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
Python-ast.h cellobject.h descrobject.h frameobject.h iterobject.h node.h py_curses.h pyfpe.h pystrcmp.h structseq.h warnings.h
Python.h ceval.h dictobject.h funcobject.h listobject.h object.h pyarena.h pyframe.h pystrhex.h symtable.h weakrefobject.h
abstract.h classobject.h dynamic_annotations.h genericaliasobject.h longintrepr.h objimpl.h pycapsule.h pyhash.h pystrtod.h sysmodule.h
asdl.h code.h enumobject.h genobject.h longobject.h odictobject.h pyconfig.h pylifecycle.h pythonrun.h token.h
ast.h codecs.h errcode.h graminit.h marshal.h opcode.h pyctype.h pymacconfig.h pythread.h traceback.h
bitset.h compile.h eval.h grammar.h memoryobject.h osdefs.h pydebug.h pymacro.h pytime.h tracemalloc.h
bltinmodule.h complexobject.h exports.h import.h methodobject.h osmodule.h pydtrace.h pymath.h rangeobject.h tupleobject.h
boolobject.h context.h fileobject.h internal modsupport.h parsetok.h pydtrace_probes.h pymem.h setobject.h typeslots.h
bytearrayobject.h cpython fileutils.h interpreteridobject.h moduleobject.h patchlevel.h pyerrors.h pyport.h sliceobject.h ucnhash.h
bytesobject.h datetime.h floatobject.h intrcheck.h namespaceobject.h picklebufobject.h pyexpat.h pystate.h structmember.h unicodeobject.h
4: check the contents of the library paths:
ls /usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib
libpython3.9.dylib pkgconfig python3.9
ls /usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin
Makefile Setup Setup.local config.c config.c.in install-sh libpython3.9.a libpython3.9.dylib makesetup python-config.py python.o
ls /usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9
LICENSE.txt binhex.py difflib.py imp.py pickletools.py site.py timeit.py
__future__.py bisect.py dis.py importlib pipes.py sitecustomize.py tkinter
__phello__.foo.py bz2.py distutils inspect.py pkgutil.py smtpd.py token.py
__pycache__ cProfile.py doctest.py io.py platform.py smtplib.py tokenize.py
_aix_support.py calendar.py email ipaddress.py plistlib.py sndhdr.py trace.py
_bootlocale.py cgi.py encodings json poplib.py socket.py traceback.py
_bootsubprocess.py cgitb.py ensurepip keyword.py posixpath.py socketserver.py tracemalloc.py
_collections_abc.py chunk.py enum.py lib-dynload pprint.py sqlite3 tty.py
_compat_pickle.py cmd.py filecmp.py lib2to3 profile.py sre_compile.py turtle.py
_compression.py code.py fileinput.py linecache.py pstats.py sre_constants.py turtledemo
_markupbase.py codecs.py fnmatch.py locale.py pty.py sre_parse.py types.py
_osx_support.py codeop.py formatter.py logging py_compile.py ssl.py typing.py
_py_abc.py collections fractions.py lzma.py pyclbr.py stat.py unittest
_pydecimal.py colorsys.py ftplib.py mailbox.py pydoc.py statistics.py urllib
_pyio.py compileall.py functools.py mailcap.py pydoc_data string.py uu.py
_sitebuiltins.py concurrent genericpath.py mimetypes.py queue.py stringprep.py uuid.py
_strptime.py config-3.9-darwin getopt.py modulefinder.py quopri.py struct.py venv
_sysconfigdata__darwin_darwin.py configparser.py getpass.py multiprocessing random.py subprocess.py warnings.py
_threading_local.py contextlib.py gettext.py netrc.py re.py sunau.py wave.py
_weakrefset.py contextvars.py glob.py nntplib.py reprlib.py symbol.py weakref.py
abc.py copy.py graphlib.py ntpath.py rlcompleter.py symtable.py webbrowser.py
aifc.py copyreg.py gzip.py nturl2path.py runpy.py sysconfig.py wsgiref
antigravity.py crypt.py hashlib.py numbers.py sched.py tabnanny.py xdrlib.py
argparse.py csv.py heapq.py opcode.py secrets.py tarfile.py xml
ast.py ctypes hmac.py operator.py selectors.py telnetlib.py xmlrpc
asynchat.py curses html optparse.py shelve.py tempfile.py zipapp.py
asyncio dataclasses.py http os.py shlex.py test zipfile.py
asyncore.py datetime.py idlelib pathlib.py shutil.py textwrap.py zipimport.py
base64.py dbm imaplib.py pdb.py signal.py this.py zoneinfo
bdb.py decimal.py imghdr.py pickle.py site-packages threading.py
5: Combine all of the above into a command that specifies the interpreter and all required paths into one thing:
cd unbound && /usr/bin/arch -x86_64 ./configure --prefix=/home/$(USER)/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis PYTHON="/usr/local/Cellar/[email protected]/3.9.9/bin/python3.9" PYTHON_SITE_PKG=$(ROOT_DIR)/.venv/lib/python3.9/site-packages PYTHON_LDFLAGS="-L/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 -L/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin -L/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib" PYTHON_CPPFLAGS="-I/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/include/python3.9" && make install
6: Add some inspections to configure and see what's really going wrong:
cat conftest.err
cat conftest.$ac_objext
cat conftest$ac_exeext
cat conftest.$ac_ext
7: See the output:
checking for python... /usr/local/Cellar/[email protected]/3.9.9/bin/python3.9
checking for the distutils Python package... yes
checking for Python include path... -I/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
checking for Python library path... -L/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 -L/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin -L/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib
checking for Python site-packages path... /Users/stitch/Documents/_webdevelopment/internetnl2/Internet.nl/.venv/lib/python3.9/site-packages
checking consistency of all components of python development environment... undef: _Py_Initialize
Undefined symbols for architecture arm64:
"_Py_Initialize", referenced from:
_main in cc-020cd3.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
8: Possible conclusions:
- There are still missing python variables
- A python version from the path is taken instead of the one specified.
- This scenario is unsupported
Any more help would be appreciated :)
edit: paths
On a fresh debian 10 install, after installing python3 from source:
I've found a bug in the configure script: it suppresses any python compilation error messages, and replacing that with an irrelevant message. Above on a mac i got a symbol warning, after the debian machine i get a 'gcc: error: libpython3.9.a: No such file or directory'.
Header files are present. I cannot find the libpython file indeed. And since there is no python3-dev on this apt, and deadsnakes does not provide it for debian 10, and there are no seperate instructions to install python3 development tools separately (i guess they would be installed when you install python3 from source, but no). Google doesnt help here.
Trying again with a fresh debian 11 vm.
The checking consistency phase should log the results of that in the config.log file. There you should be able to find the compiler command that was executed and the failing results from it. Like file not found errors.
The PYTHON_LDFLAGS also needs to have -lpython in there. This is likely why it does not link. Put that after the directory -L contents. The second time it does not link with libpython3.9.a, I am thinking you did not set PYTHON_LDFLAGS, otherwise I am unsure how it would reference that file.
Also set the variable PYTHON_LIBDIR to the name of one directory, one with python.dylib in there for runtime.
There we are:
.unbound-x86-3.8:
# For m1 users:
# arch -x86_64 /bin/bash
# /usr/local/Homebrew/bin/brew install [email protected]
# brew unlink [email protected] && brew link [email protected]
# /usr/local/Homebrew/bin/brew install libevent
# /usr/local/Homebrew/bin/brew install hiredis
rm -rf unbound
git clone https://github.com/internetstandards/unbound
cd unbound && /usr/bin/arch -x86_64 ./configure --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis PYTHON="/usr/local/Cellar/[email protected]/3.8.12_1/bin/python3.8" PYTHON_SITE_PKG=$(ROOT_DIR)/.venv/lib/python3.8/site-packages PYTHON_LDFLAGS="-L/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8 -L/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin -L/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib -lpython3.8" PYTHON_CPPFLAGS="-I/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/include/python3.8" PYTHON_LIBDIR="/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib" && make install
touch .unbound-x86-3.8
Reopening it because of the same miserable experience on a random debian machine where a python 3.7 is needed instead of 3.9.
I've installed python 3.7 via source and it's missing the library dirs. There is little instruction on how to get it or where to start and searching is not really delvering.
Seriously, the python dependencies are a hurdle every installation. Can you make this process any easier or better? It's just awful.
Usually the python dependencies are installed from the package management system, and I have not had other complaints on how it could be better. The configure now wants to use the python settings for its directories, or you can set them with flags to configure.
So the python is needed that is not the one from the package management system. So you need to select it somehow with configure flags. Configure outputs python checks, and those also have entries in config.log. Does it make it through that and compile or fail at build stage? What is wrong?
That's weird, i'm wondering how nobody runs into this.
When the python deps are installed from the package manager it all works fine, you get the development headers and whatnot and simply compiling and running works. It goes wrong when trying to install a custom python version, when the software you're working on wants an older version.
So under debian 11, there is no trivial way to get something like "libpython3.7-dev" and so it seems that getting unbound compiled for python3.7 is impossible. Perhaps you know better?
I have not really experience with the matter. I would assume from other packages that you should install the python3.7 in some other destination and then set PYTHON_VERSION=3.7 on the ./configure command line. This approach has worked for me. Make sure that the python3.7 binary is in the executable lookup path when configure runs.
Why not use pythonX.Y-config script to obtain the information? I suspect I know the answer: because unbound's Python integration predates the pythonX.Y-config scripts.
However, using that tool with the right switches works to configure unbound, for any recent X.Y version of Python:
./configure [other options] \
--with-pythonmodule \
PYTHON_VERSION="X.Y" \
PYTHON_LDFLAGS="$(pythonX.Y-config --ldflags --embed)" \
PYTHON_CPPFLAGS="$(pythonX.Y-config --includes)"
Note: the X.Y values above should be replaced by your actual Python version.
I used the --embed switch for pythonX.Y-config, it's the crucial part here. It produces the LDFLAGS values for embedding the interpreter, meaning it adds -lpythonX.Y to the flags. This value is built from sysconfig.get_config_var("VERSION") and sys.abiflags (for sys.abiflags you can also use sysconfig.get_config_var("ABIFLAGS")).
The unbound-project supplied acx_python.m4 instead uses sysconfig.get_config_var('BLDLIBRARY'), which is libpythonX.Y.a (so including the lib prefix and .a extension), and the script doesn't then prefix this with -l, and so is doomed to fail.
@mjpieters
Thank you for this!! Using those python-config options allowed me to compile on Ubuntu 22.04 w/ Python 3.10.
./configure --prefix=/usr --includedir=\${prefix}/include --infodir=\${prefix}/share/info --mandir=\${prefix}/share/man --localstatedir=/var --runstatedir=/run --sysconfdir=/etc --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --with-libevent --with-libhiredis --with-libnghttp2 --with-pidfile=/run/unbound.pid --with-pythonmodule --with-pyunbound --with-rootkey-file=/var/lib/unbound/root.key --disable-dependency-tracking --disable-flto --disable-maintainer-mode --disable-option-checking --disable-rpath --disable-silent-rules --enable-cachedb --enable-dnstap --enable-subnet --enable-systemd --enable-tfo-client --enable-tfo-server PYTHON_VERSION=3.10 PYTHON_LDFLAGS="$(python3.10-config --ldflags --embed)" PYTHON_CPPFLAGS="$(python3.10-config --includes)" CFLAGS="-O2"
