f90wrap icon indicating copy to clipboard operation
f90wrap copied to clipboard

Windows missing dll error when building and running examples on Windows with Python 3.8>

Open sbland opened this issue 4 years ago • 16 comments
trafficstars

Since Python 3.8 dlls are only loaded from a trusted location on Windows https://docs.python.org/3/whatsnew/3.8.html#ctypes This brakes building and running f90wrap on Windows due to the following error: ImportError: DLL load failed while importing sizeof_fortran_t: The specified module could not be found. This can be fixed by adding the dll location using os.add_dll_directory("PATH_TO_f90WRAP/f90wrap.libs"). I'm not sure what the solution is to fixing this in f90wrap but I'll take a look.

sbland avatar May 07 '21 15:05 sbland

Thanks for reporting. I don't have access to a Windows machine to test on, but will be happy to receive a PR to fix this.

jameskermode avatar May 07 '21 15:05 jameskermode

Made some progress on this by adding the below to f90wrap/__init__.py.

import os

if os.name =="nt":
    os.add_dll_directory(f'{os.path.dirname(os.path.realpath(__file__))}/.libs')

This resolves the missing dll issue and simple examples will build. Unfortunately some examples now have an undefined reference to '__precision_MOD_five' error when trying to run f2py-f90wrap. See full error below:

...
gfortran.exe:f90: f90wrap/f90wrap_aa2_defineAllProperties.f90
gfortran.exe:f90: f90wrap/f90wrap_toplevel.f90
copying .\libsrc.a -> .\Release\src.lib
C:\Apps\mingw64\bin\gfortran.exe -Wall -g -Wall -g -shared ~\f90wrap\examples\example2\Release\f90wrap\f90wrap_aa0_typelist.o ~\f90wrap\examples\example2\Release\f90wrap\f90wrap_aa1_modules.o ~\f90wrap\examples\example2\Release\f90wrap\f90wrap_aa2_defineAllProperties.o ~\f90wrap\examples\example2\Release\f90wrap\f90wrap_toplevel.o -LC:\Apps\mingw64\lib\gcc\x86_64-w64-mingw32\7.2.0 -L~\f90wrap\venv\libs -LC:\~\AppData\Local\Programs\Python\Python38\libs -LC:\~\AppData\Local\Programs\Python\Python38 -L~\f90wrap\venv\PCbuild\amd64 -o .\Release\.libs\libf90wrap_.HUF45REHGPWYBYV53E2NYKMSFCRWWWJV.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,.\Release\libf90wrap_.HUF45REHGPWYBYV53E2NYKMSFCRWWWJV.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64
~\f90wrap\examples\example2\Release\f90wrap\f90wrap_aa0_typelist.o:f90wrap_aa0_typelist.f90:(.rdata$.refptr.__precision_MOD_five[.refptr.__precision_MOD_five]+0x0): undefined reference to `__precision_MOD_five'
~\f90wrap\examples\example2\Release\f90wrap\f90wrap_aa0_typelist.o:f90wrap_aa0_typelist.f90:(.rdata$.refptr.__precision_MOD_in2ft[.refptr.__precision_MOD_in2ft]+0x0): undefined reference to `__precision_MOD_in2ft'

I'll take another look into this later and see if I can make a smaller example.

sbland avatar May 12 '21 16:05 sbland

Nearly there! This is pushing my limited understanding of the compilers but it seems to work. Adding the original .f90 files to the f2py-f90wrap call results in successful compilation when just a single file is used. f2py-f90wrap --fcompiler=gfortran --build-dir . -c -m _pipeline -L. -lsrc f90wrap_demo.f90 to f2py-f90wrap --fcompiler=gfortran --build-dir . -c demo.f90 -m _pipeline -L. -lsrc f90wrap_demo.f90 demo.f90:

module demo
    implicit none

    TYPE :: DemoType
        Integer :: nL
    end type DemoType
    public :: hello
    contains
    ! This function caused build fail without above change
    function hello(val_in) result(outval)
        REAL, intent(in):: val_in
        REAL :: outval
        outval = val_in + 1.0
    end function
end module demo

Including more than 1 file caused a seg fault!

sbland avatar May 13 '21 09:05 sbland

OK, looks good. Usually you would compile demo.f90 yourself separately (e.g. with gfortran -c demo.f90 -o demo.o) and include the resulting object file demo.o on the f2py-f90wrap command line, or combine a number of .o files into a static library (.a in Linux, not sure how this works in Windows).

jameskermode avatar May 13 '21 09:05 jameskermode

That's what I had been doing for linux but in Windows it was having some issues. I'll take another look later.

sbland avatar May 13 '21 09:05 sbland

@sbland would you mind sharing your compiler list and build tools (MSVC? gcc through MSYS/mingw/something else) for windows? I'm trying something out with all three, found your hints super useful.. if I hit upon a solution I can report here too

nenanth avatar Aug 08 '21 23:08 nenanth

I got f90wrap to work on Windows 10 (64 bit, home edition) with Anaconda Python 3.8.8, numpy version 1.20.1 and f90wrap 0.2.7; gfortran through mingw-w64 (v8.1.0). I built f90wrap with python setup.py install after telling distutils.cfg to use mingw32 (and not the microsoft visual c/c++ compiler)

nenanth avatar Aug 09 '21 07:08 nenanth

Note that gfortran is available through conda-forge via m2w64-gcc-fortran, so you should not need to install those outside your conda environment (there is also m2w64-gcc and m2w64-toolchain and various others)

bilderbuchi avatar Aug 09 '21 09:08 bilderbuchi

Note that gfortran is available through conda-forge via m2w64-gcc-fortran, so you should not need to install those outside your conda environment (there is also m2w64-gcc and m2w64-toolchain and various others)

Thanks @bilderbuchi - I'll try it out.

nenanth avatar Aug 09 '21 17:08 nenanth

@bilderbuchi It seems the package contains some older versions of the compilers; is it up-to-date with what ships with the other toolchains? The conda package version seems to suggest gcc v5.x

nenanth avatar Aug 09 '21 17:08 nenanth

Yeah, no, it's 5.3.0, unfortunately not the freshest, but luckily enough for my Fortran codebase. As this gcc is underpinning much of the scientific Python ecosystem it's not easy to upgrade, apparently, but there are issues about that, see e.g. https://github.com/conda-forge/fpm-feedstock/issues/4 https://github.com/conda-forge/conda-forge.github.io/issues/1044

bilderbuchi avatar Aug 09 '21 18:08 bilderbuchi

Also https://github.com/conda-forge/conda-forge.github.io/issues/961

bilderbuchi avatar Aug 09 '21 18:08 bilderbuchi

@sbland did you try the mingw-w64 toolchain? Hope it resolved your issue. Looks like Windows 11 allows for WSL to be installed as an application through the Microsoft store, so you could try that if your IT policy is permissive enough.

nenanth avatar Oct 13 '21 06:10 nenanth

i tried setup.py install with anaconda 3.8.8 on a windows machine today; latest f90wrap does not need the patch in init.py. This issue can be closed?

nenanth avatar Mar 23 '22 03:03 nenanth

@sbland Did you compile the shared library on Windows 10 while you define and use "user defined data type"? Your example code went well on my Windows 10. I compiled the *.pyd file and *.dll file and imported into Python and got correct results. However, while I use the "user defined data type", the compiler will assume they are "void" and give me the error "getctype: No C-type found in "{'typespec': 'type', 'typename': 'real_array'}", assuming void." The code I tested is from "f90wrap/examples/arrays_in_derived_types_issue50" test.f90 ` module module_test

    type real_array
        real, dimension(6) :: item
    end type real_array

    contains

    subroutine testf(x)
        implicit none
        type(real_array) :: x

        print*, "This is received in fortran : ", x%item
        x%item(4) = 4
        print*, "This is sent back to python : ", x%item
    end subroutine testf

end module module_test`

Do you have any suggestions? Thank you.

Aaron2014 avatar Feb 01 '23 16:02 Aaron2014