numpy
f2py fails to link program on windows using v1.26.0 - works on earlier versions of numpy
Describe the issue:
I have an old fortran program that I access from python using f2py. I have used this successfully on both linux and windows. I have moved successfully to 1.26 on linux but get a link error (1120 unresolved externals) on windows 11. I replicated the problem using a basic fibonacci fortran program (below)
Reproduce the code example:
subroutine fib(a, n)
use iso_c_binding
integer(c_int), intent(in) :: n
integer(c_int), intent(out) :: a(n)
do i = 1, n
if (i .eq. 1) then
a(i) = 0.0d0
elseif (i .eq. 2) then
a(i) = 1.0d0
else
a(i) = a(i - 1) + a(i - 2)
end if
end do
end
Error message:
(aov) C:\fib_test>f2py -c fib1.f90 -m fib1 --backend meson
Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.htmlReading fortran codes...
Reading file 'fib1.f90' (format:free)
Post-processing...
Block: fib1
Block: fib
In: :fib1:fib1.f90:fib
get_useparameters: no module iso_c_binding info used by fib
Applying post-processing hooks...
character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
Building module "fib1"...
Generating possibly empty wrappers"
Maybe empty "fib1-f2pywrappers.f"
Constructing wrapper function "fib"...
a = fib(n)
Wrote C/API module "fib1" to file ".\fib1module.c"
The Meson build system
Version: 1.2.1
Source dir: C:\Users\keith\AppData\Local\Temp\tmp679okcip
Build dir: C:\Users\keith\AppData\Local\Temp\tmp679okcip\bbdir
Build type: native build
Project name: fib1
Project version: 0.1
Fortran compiler for the host machine: ifx (intel-llvm-cl 2022.1.0)
Fortran linker for the host machine: xilink.exe xilink 2022.1.0
C compiler for the host machine: cl (msvc 19.29.30146 "Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30146 for x64")
C linker for the host machine: link link 14.29.30146.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Users\keith\anaconda3\envs\aov\python.exe)
Run-time dependency python found: YES 3.12
Build targets in project: 1
Found ninja-1.10.2 at C:\Users\keith\anaconda3\envs\aov\Library\bin\ninja.EXE
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\Users\keith\anaconda3\envs\aov\Library\bin\ninja.EXE -C C:/Users/keith/AppData/Local/Temp/tmp679okcip/bbdir
ninja: Entering directory `C:/Users/keith/AppData/Local/Temp/tmp679okcip/bbdir'
[4/6] Compiling Fortran object fib1.cp312-win_amd64.pyd.p/fib1-f2pywrappers.f.obj
../fib1-f2pywrappers.f: remark #5133: The input stream is empty
[6/6] Linking target fib1.cp312-win_amd64.pyd
FAILED: fib1.cp312-win_amd64.pyd
"link" /MACHINE:x64 /OUT:fib1.cp312-win_amd64.pyd fib1.cp312-win_amd64.pyd.p/fib1.f90.obj fib1.cp312-win_amd64.pyd.p/fib1module.c.obj fib1.cp312-win_amd64.pyd.p/fib1-f2pywrappers.f.obj fib1.cp312-win_amd64.pyd.p/42bc432c09fd0a50dc493f9b8a0dabb66e5dad74_.._.._f2py_src_fortranobject.c.obj "/nologo" "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:fib1.cp312-win_amd64.lib" "-shared" "C:\Users\keith\anaconda3\envs\aov\libs\python312.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : warning LNK4044: unrecognized option '/shared'; ignored
Creating library fib1.cp312-win_amd64.lib and object fib1.cp312-win_amd64.exp
fib1module.c.obj : error LNK2001: unresolved external symbol fib_
fib1.cp312-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "C:\Users\keith\anaconda3\envs\aov\Scripts\f2py-script.py", line 10, in <module>
sys.exit(main())
^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\f2py2e.py", line 732, in main
run_compile()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\f2py2e.py", line 705, in run_compile
builder.compile()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 131, in compile
self.run_meson(self.build_dir)
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 124, in run_meson
raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['meson', 'compile', '-C', 'bbdir']' returned non-zero exit status 1.
Python and NumPy Versions:
python 3.12.3 Numpy 1.26.0
Runtime Environment:
(aov) C:\Users\keith\temp\fib_test>python -c "import numpy; print(numpy.show_runtime())" [{'numpy_version': '1.26.0', 'python': '3.12.3 | packaged by Anaconda, Inc. | (main, Apr 19 2024, ' '16:41:55) [MSC v.1916 64 bit (AMD64)]', 'uname': uname_result(system='Windows', node='LENNY2', release='11', version='10.0.22631', machine='AMD64')}, {'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'], 'found': ['SSSE3', 'SSE41', 'POPCNT', 'SSE42', 'AVX', 'F16C', 'FMA3', 'AVX2'], 'not_found': ['AVX512F', 'AVX512CD', 'AVX512_SKX', 'AVX512_CLX', 'AVX512_CNL', 'AVX512_ICL']}}, {'filepath': 'C:\Users\keith\anaconda3\envs\aov\Library\bin\mkl_rt.2.dll', 'internal_api': 'mkl', 'num_threads': 8, 'prefix': 'mkl_rt', 'threading_layer': 'intel', 'user_api': 'blas', 'version': '2023.1-Product'}] None
Context for the issue:
A user has upgraded to python 3.12 and cannot use our product
Can you try again with numpy 1.26.4? Numpy 1.26.3 had a number of f2py-related fixes.
Hi Nathan
Yes – certainly. I have updated to 1.26.4 and I get a different set of errors:-
(aov) C:\Users\keith\temp\fib_test>f2py -c fib1.f90 -m fib1 --backend meson
Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.html
Reading fortran codes...
Reading file 'fib1.f90' (format:free)
Post-processing...
Block: fib1
Block: fib
In: :fib1:fib1.f90:fib
get_useparameters: no module iso_c_binding info used by fib
Applying post-processing hooks...
character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
Building module "fib1"...
Generating possibly empty wrappers"
Maybe empty "fib1-f2pywrappers.f"
Constructing wrapper function "fib"...
a = fib(n)
Wrote C/API module "fib1" to file ".\fib1module.c"
The Meson build system
Version: 1.2.1
Source dir: C:\Users\keith\AppData\Local\Temp\tmpz_nfwcsm
Build dir: C:\Users\keith\AppData\Local\Temp\tmpz_nfwcsm\bbdir
Build type: native build
Project name: fib1
Project version: 0.1
Fortran compiler for the host machine: ifx (intel-llvm-cl 2022.1.0)
Fortran linker for the host machine: xilink.exe xilink 2022.1.0
C compiler for the host machine: cl (msvc 19.29.30146 "Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30146 for x64")
C linker for the host machine: link link 14.29.30146.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Traceback (most recent call last):
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\mesonmain.py", line 194, in run
return options.run_func(options)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\msetup.py", line 358, in run
app.generate()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\msetup.py", line 183, in generate
return self._generate(env, capture, vslite_ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\msetup.py", line 228, in _generate
intr.run()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreter\interpreter.py", line 2997, in run
super().run()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 164, in run
self.evaluate_codeblock(self.ast, start=1)
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 190, in evaluate_codeblock
raise e
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 182, in evaluate_codeblock
self.evaluate_statement(cur)
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 198, in evaluate_statement
self.assignment(cur)
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 635, in assignment
value = self.evaluate_statement(node.value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 200, in evaluate_statement
return self.method_call(cur)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 550, in method_call
res = obj.method_call(method_name, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreter\interpreterobjects.py", line 811, in method_call
ret = method(state, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 109, in wrapped
ret = f(*wrapped_args, **wrapped_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 277, in wrapper
return f(*nargs, **wrapped_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 596, in wrapper
return f(*wrapped_args, **wrapped_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\modules\python.py", line 437, in find_installation
python = self._find_installation_impl(state, display_name, name_or_path, required)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\modules\python.py", line 393, in _find_installation_impl
if python.sanity(state):
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\modules\python.py", line 74, in sanity
ret = super().sanity()
^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\dependencies\python.py", line 116, in sanity
p, stdout, stderr = mesonlib.Popen_safe(cmd)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\utils\universal.py", line 1483, in Popen_safe
p, o, e = Popen_safe_legacy(args, write=write, stdin=stdin, stdout=stdout, stderr=stderr, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\mesonbuild\utils\universal.py", line 1505, in Popen_safe_legacy
p = subprocess.Popen(args, universal_newlines=False, close_fds=False,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\keith\anaconda3\envs\aov\Lib\subprocess.py", line 1538, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified
ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.
Traceback (most recent call last):
File "C:\Users\keith\anaconda3\envs\aov\Scripts\f2py-script.py", line 10, in <module>
sys.exit(main())
^^^^^^
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\f2py2e.py", line 766, in main
run_compile()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\f2py2e.py", line 738, in run_compile
builder.compile()
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 178, in compile
self.run_meson(self.build_dir)
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 171, in run_meson
self._run_subprocess_command(setup_command, build_dir)
File "C:\Users\keith\anaconda3\envs\aov\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 167, in _run_subprocess_command
subprocess.run(command, cwd=cwd, check=True)
File "C:\Users\keith\anaconda3\envs\aov\Lib\subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['meson', 'setup', 'bbdir']' returned non-zero exit status 2.
Regards
keith
EDIT (seberg): tied to fix formatting, but emails just disable any markdown stuff, so can only remove the spurious newlines...
@HaoZeke Thoughts?
Thanks for the ping. Since this is windows and related to not finding an executable, it might be fixed by https://github.com/numpy/numpy/pull/26248
@keithini could you try to reproduce this bug on main after #26248?
Beyond that I will try to reproduce this on a windows machine if I can
Hi I was not exactly sure what I needed to do to implement "main after #26248" and so I started again with a clean environment and installed numpy 1.26.4. When I tried f2py I got the same errors:
C:\Users\keith\temp\fib_test>f2py -c fib1.f90 -m fib1 --backend meson Using meson backend Will pass --lower to f2py See https://numpy.org/doc/stable/f2py/buildtools/meson.html Reading fortran codes... Reading file 'fib1.f90' (format:free) Post-processing... Block: fib1 Block: fib In: :fib1:fib1.f90:fib get_useparameters: no module iso_c_binding info used by fib Applying post-processing hooks... character_backward_compatibility_hook Post-processing (stage 2)... Building modules... Building module "fib1"... Generating possibly empty wrappers" Maybe empty "fib1-f2pywrappers.f" Constructing wrapper function "fib"... a = fib(n) Wrote C/API module "fib1" to file ".\fib1module.c" The Meson build system Version: 1.3.1 Source dir: C:\Users\keith\AppData\Local\Temp\tmpjd_uuokj Build dir: C:\Users\keith\AppData\Local\Temp\tmpjd_uuokj\bbdir Build type: native build Project name: fib1 Project version: 0.1 Fortran compiler for the host machine: ifort (intel-cl 2021.6.0) Fortran linker for the host machine: xilink.exe xilink 2021.6.0 C compiler for the host machine: cl (msvc 19.29.30146 "Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30146 for x64") C linker for the host machine: link link 14.29.30146.0 Host machine cpu family: x86_64 Host machine cpu: x86_64 Traceback (most recent call last): File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\mesonmain.py", line 194, in run return options.run_func(options) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\msetup.py", line 358, in run app.generate() File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\msetup.py", line 181, in generate return self._generate(env, capture, vslite_ctx) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\msetup.py", line 226, in _generate intr.run() File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreter\interpreter.py", line 3004, in run super().run() File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 172, in run self.evaluate_codeblock(self.ast, start=1) File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 198, in evaluate_codeblock raise e File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 190, in evaluate_codeblock self.evaluate_statement(cur) File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 208, in evaluate_statement self.assignment(cur) File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 644, in assignment value = self.evaluate_statement(node.value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 210, in evaluate_statement return self.method_call(cur) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\interpreterbase.py", line 559, in method_call res = obj.method_call(method_name, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreter\interpreterobjects.py", line 830, in method_call ret = method(state, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 108, in wrapped ret = f(*wrapped_args, **wrapped_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 260, in wrapper return f(*nargs, **wrapped_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\interpreterbase\decorators.py", line 579, in wrapper return f(*wrapped_args, **wrapped_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\modules\python.py", line 515, in find_installation python = self._find_installation_impl(state, display_name, name_or_path, required) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\modules\python.py", line 471, in _find_installation_impl if python.sanity(state): ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\modules\python.py", line 83, in sanity ret = super().sanity() ^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\dependencies\python.py", line 120, in sanity p, stdout, stderr = mesonlib.Popen_safe(cmd, env=env) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\utils\universal.py", line 1526, in Popen_safe p, o, e = Popen_safe_legacy(args, write=write, stdin=stdin, stdout=stdout, stderr=stderr, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\site-packages\mesonbuild\utils\universal.py", line 1548, in Popen_safe_legacy p = subprocess.Popen(args, universal_newlines=False, close_fds=False, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\keith\anaconda3\envs\aov2\Lib\subprocess.py", line 1026, in init self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\keith\anaconda3\envs\aov2\Lib\subprocess.py", line 1538, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [WinError 2] The system cannot find the file specified
ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.
Traceback (most recent call last):
File "C:\Users\keith\anaconda3\envs\aov2\Scripts\f2py-script.py", line 10, in
Same issue on windows/conda in 1.26.4 for me. I'm able to build the examples in my python 3.8 environment. In the meantime is there any issue with just using 3.8? Is there a different python/numpy version you would recommend?
