WIP: Add windows build

[Jason23347]

Todos

• [x] Make socket compatable
• [x] Build with Openssl
• [x] Test with wine on ubuntu
• [x] Test on Windows (win10)
• [ ] Header checks and function checks in configure.ac should be modified
• [x] pkg-config doesn't work

Known problems

• SSL certificate verification alwasy fail.

Build from this branch

Download and compile mingw

On ubuntu it is easy:

sudo apt install mingw-w64

Download and compile openssl against mingw

I chose 1.1.1 release on github There was an article about building openssl against mingw on linux: https://marc.xn--wckerlin-0za.ch/computer/cross-compile-openssl-for-windows-on-linux

cd /path/to/openssl
./Configure mingw shared --cross-compile-prefix=x86_64-w64-mingw32-

before making, 3 lines should be added to file include/openssl/x509v3.h, just after #define HEADER_X509V3_H:

#ifdef X509_NAME
#undef X509_NAME
#endif

Build Axel

autoconf -i
SSL_CFLAGS=-I/path/to/openssl/include \
SSL_LIBS="-L/path/to/openssl -lssl -lcrypto" \
./configure --host=x86_64-w64-mingw32 --disable-Werror

This attachment file is an exutable file of windows environment.

Closes #108

[Jason23347]

@ismaell, please take a look at the code I changed in tcp.c, http.c, etc. I'm not sure if these changes are acceptable. Also I have doubts whether if-condition #ifdef HAVE_SSL in compat-win32.h is acceptable.

[ismaell]

The proper way to add support for windows would be to move the OS test to the beginning of the configure script and avoid running the posix-specifc function tests for windows, instead including a compatibility library right away with an drop-in implementation.

[ismaell]

• Can WSAEWOULDBLOCK be translated to EINPROGRESS? is it really functionally equivalent in the test in tcp_connect?
• Why can't we provide emulation for fcntl instead of implementing SET_SOCK_BLOCK?
• Wrapping read and write and translating the socket fds should be also possible, albeit more complicated, if it can be done sensibly I would go for that.

[Jason23347]

• Can WSAEWOULDBLOCK be translated to EINPROGRESS? is it really functionally equivalent in the test in tcp_connect?

According to windows doc, I think they are functionally equivalent. But I'm not sure where should I translate WSA errors to errno.

• Why can't we provide emulation for fcntl instead of implementing SET_SOCK_BLOCK?

Ok. I will do the rebase for it.

• Wrapping read and write and translating the socket fds should be also possible, albeit more complicated, if it can be done sensibly I would go for that.

send and recv can be used for different systems, I have a question that why read and write should be kept in tcp_read and tcp_write. SOCKET type is unsigned. so changes from "-1" to INVALID_SOCKET seems necessary.

[ismaell]

I'm not making changes to correct code just because Windows fails to do the right thing. It's up to the compatibility layer for windows to fix any platform differences, transparently.

[ismaell]

Does the following work?

sh configure --with-ssl=/path/to/openssl --host=x86_64-w64-mingw32

That would be better than specifying SSL_LIBS and SSL_CFLAGS.

[Jason23347]

Does the following work?

sh configure --with-ssl=/path/to/openssl --host=x86_64-w64-mingw32
```is

That would be better than specifying `SSL_LIBS` and `SSL_CFLAGS`.

I have read the code in configure.ac, it seems with_ssl can only be set to yes/no. Originally there's no need to specify SSL_LIBS and SSL_CFLAGS, pkg-config could take care of modules using the PKG_CHECK_MODULES macro. The reason why they were specified is that OpenSSL could not be found by pkg-config automatically cause I didn't do the "make install" when I built OpenSSL. And it is not difficult to build axel (MinGW) using pkg-config with the libssl.pc file in OpenSSL path.

In all, this is not a problem with Axel. However, it might be necessary to add descriptions of pkg-config to the building guide.

[ismaell]

Indeed, I know it's only needed when libssl is in a non-standard path... I thought I had implemented taking a path on that arg before... I'll do that.

[ismaell]

I think documenting all the little options you have to configure a package is a little bit overkill.

[Jason23347]

I was considering the condition that you want to build from the master branch instead of the release tarballs, in which case you might have difficulties configuring OpenSSL. But it becomes less necessary that documenting for win32 build if the --with-ssl option could have a path arg.

[ismaell]

https://github.com/axel-download-accelerator/axel/issues/325

[ismaell]

@Jason23347 I've implemented it, but I'm not sure it works on Windows... you still need to install openssl/libressl somewhere, so that the .pc files are found at the right place, but not necessarily in a standard path.

[Jason23347]

The code in #325 worked fine if the suffix "/lib/pkg-config" had been removed. The file libssl.pc could be found right in the OpenSSL directory by default. Now, I can make a win32 build on Ubuntu PC with command

./configure --host=x86_64-w64-mingw32 --disable-Werror --with-ssl=/home/jason/openssl-OpenSSL_1_1_1

@ismaell Should I do the rebase to move all my commits afterward current master HEAD?

[ismaell]

Does the make install on openssl produce different results when compiling for Windows? is /home/jason/openssl-OpenSSL_1_1_1 an installation path??

[Jason23347]

By default, the make install with copy the libssl.pc and openssl.pc file to /usr/local/lib/pkg-config just as usual. I wonder if there is a necessity about building Axel with OpenSSL built but not installed...

[ismaell]

I wonder if there is a necessity about building Axel with OpenSSL built but not installed...

No, that's always the wrong thing to do, you must install it somewhere, not necessarily a global place...

[Jason23347]

Ok, I got it. It works fine while the installation path is configured.

[ismaell]

I'm just going to go ahead and add the _WIN32_WINNT flag to configure, do you mind to check it works?

[Jason23347]

I don't know why but the variable $host_os is set to mingw32 and cannot match the pattern *-mingw32... It works fine if *-mingw32 is changed to mingw32.

[Jason23347]

Tonight I will do the rebase to change SET_SOCK_BLOCK back to fcntl. But I still have no idea about the header checking in configure.ac... Some of the header files and functions does not exist in mingw. @ismaell How should I modify the AC_CHECK_* part?

[ismaell]

@Jason23347 sorry for the delay replying.

Indeed, some tests need to be changed. Any header that might not be present, e.g. a POSIX header not available in MinGW, could be checked conditionally this way:

AS_CASE(["$host_os"],
  [*mingw32], [
    # Provide a replacement or AC_DEFINE some flag here
  ], [
    AC_CHECK_HEADERS([ \
        header1.h \
        header2.h \
        ...
    ],, [
      AC_MSG_ERROR([$ac_header is required.])
    ])
])

Ideally, an overlay of headers and replacement functions are provided so that the rest of the code is valid. Having #ifdefs is a lot harder to maintain over the years and platform changes than a compatibility layer.

I moved the OS test up in the configure script so that $host_os can be use about anywhere.

[Jason23347]

Thanks a lot for replying,I will change the header check in this week. @ismaell Should I create another PR, or modify this branch?

[Jason23347]

@ismaell It seems all worked fine except the function check of nanosleep failed, which is not provided by MinGW. Should we implement the function or use other function instead?

[ismaell]

Tonight I will do the rebase to change SET_SOCK_BLOCK back to fcntl. But I still have no idea about the header checking in configure.ac... Some of the header files and functions does not exist in mingw. @ismaell How should I modify the AC_CHECK_* part?

Hmm, because it's a non-POSIX system, it should perhaps be handled before calling AC_CHECK_FUNCS, like this:

# OS-specific issues
AS_CASE(["$host_os"],
 [*mingw32], [
   AC_LIBOBJ([nanosleep])
 ])

You can find an implementation in gnulib.

[avdaredevil]

Hey @Jason23347 @ismaell any headway on this? It'd be nice to get this in as axel initially used to be windows compatible. Thanks :)

[ismaell]

@avdaredevil in slowmotion... I don't use Windows. It is a good candidate for a bounty.

[avdaredevil]

How close is this PR to being done, maybe I can sink a Saturday on this? Also what type of tooling is required to build this project?

[ismaell]

@avdaredevil you'll need MinGW, OpenSSL, WinSock and a pthreads implementation for starters. Cross-compiling is probably easier than building on Windows, but you can try either way.

[ismaell]

I'm going to modify configure to take the OpenSSL installation prefix in a SSL_PREFIX variable in the near future.

[ismaell]

@avdaredevil BTW; there's a msys2 package you can use as a starting point.