./configure fails with Unicode build

[dEajL3kA]

I did this

I am trying to build curl with Unicode support on Windows (using MinGW/MSYS2).

In order to enable Unicode, I added options -munciode and -mconsole to the CFLAGS explicitely, because apparently ./configure doesn't have a dedicated --enable-unicode switch. Or did I miss something?

Unforntunately, this caused ./configure to fail with "No working C compiler" error. My analysis of the config.log showed that ./configure uses a lot of "test" programs that have main() hardcoded, but should be using wmain() when Unicode is used! In the actual curl source codes we already do have the required #ifdefs to use either main() or wmain() as needed. Only the ./configure has problem! My workaround was to replace all occurences of "main" in ./configure with "wmain", using a simple sed command. This does work. After patching configure, the Unicode version of curl compiled without any problem!

sed -i -E 's/\bmain[[:space:]]*\(([^\(\)]*)\)/wmain(\1)/g' configure

But this workaround is rather cumbersome and other people may run into the same issue. So I think it would be much preferable to have an --enable-unicode switch that not only addes -munciode and -mconsole to the CFLAGS automatically but also uses the "correct" main function in the "test" programs. At least make ./configure work with -munciode -mconsole options.

I expected the following

CFLAGS="-municode -mconsole" ./configure && make should "just work" without the need to patch configure.

Even better, ./configure --enable-unicode && make should be available.

curl/libcurl version

7.77.0

operating system

Windows 10, using latest MinGW/MSYS2 compiler + build environment

MINGW32_NT-10.0-19042 3.2.0-340.x86_64 2021-05-24 19:32 UTC x86_64 Msys
gcc version 10.3.0 (Rev2, Built by MSYS2 project)

[dEajL3kA]

EDIT: Better solution exists. Please see here!

---

Possible workaround, if we don't want to patch the configure script, would be using a tiny "shim" library that implements the wmain() function, in order to fix the linker error about missing wmain (or wWinMain) entry point, and that does nothing but invoking the "real" main() function. Libary needs to be added to LIBS environment variable when invoking ./configure.

libwmain.c:

extern int main(int argc, char *argv[]);
int wmain(void) {
	char *args[] = { "a.exe", (void*)0 };
	return main(1, (char**)args);
}

Makefile:

libwmain.a: libwmain.o
	$(AR) rcs libwmain.a libwmain.o
libwmain.o: libwmain.c
	$(CC) -march=i386 -Os -o libwmain.o -c libwmain.c

To build curl: CFLAGS="-munciode -mconsole" LIBS="-lwmain" ./configure

Note: In my test, this did not collide with curl's own wmain() method, but did fix the conftests.

[jay]

Can you use -DUNICODE -D_UNICODE instead

[dEajL3kA]

Can you use -DUNICODE -D_UNICODE instead

Sorry, but nope. Or I am missing something? 😕

• According to my tests, -municode -mconsole already implies -DUNICODE -D_UNICODE, so explicitely adding -DUNICODE -D_UNICODE to CFLAGS in addition to -municode -mconsole does not change anything for me.

• Setting only -DUNICODE -D_UNICODE in CFLAGS but not -municode -mconsole causes MinGW/GCC to still link against main() rather than wmain() – this I have just verified! As a result, ./configure does succeed without additional hacks, yes. But, in the end, the linking of curl.exe fails with an undefined reference error – which is expected, because with -D_UNICODE we are telling tool_main.c to generate wmain() instead of main(), but at the same time MinGW/GCC is expecting a main() function due to the lack of -municode -mconsole switches. Hence, setting -DUNICODE -D_UNICODE alone is not sufficient 😞

This was tested with latest MinGW/GCC: gcc version 10.3.0 (Rev2, Built by MSYS2 project)

Best regards.

[jay]

It seems this is the way curl-for-win does it:

  CURL_CFLAG_EXTRAS="${CURL_CFLAG_EXTRAS} -DUNICODE -D_UNICODE"
  CURL_LDFLAG_EXTRAS_EXE="${CURL_LDFLAG_EXTRAS_EXE} -municode"

So we'd have to apply -municode only for the curl tool?

/cc @vszakats

[dEajL3kA]

So we'd have to apply -municode only for the curl tool?

Yes, I think so. But it seems that CURL_LDFLAG_EXTRAS_EXE does nothing for me. I did a full-text search on both, configure and the resulting Makefile, but could not find the identifier CURL_LDFLAG_EXTRAS_EXE at all. Does it even exist in "vanilla" curl?

However, what does work for me is:

CFLAGS="-DUNICODE -D_UNICODE" ./configure
make curl_LDFLAGS="-all-static -municode -mconsole"

So that probably is the way to go. Still, figuring this out was wearisome. Maybe add a ./configure option for Unicode build?

Thank you and best regards.

(BTW: What is the difference between CURL_CFLAG_EXTRAS and CFLAGS variables?)

[jay]

Maybe add a ./configure option for Unicode build?

Seems like a good idea.

(BTW: What is the difference between CURL_CFLAG_EXTRAS and CFLAGS variables?)

I think it was added to pass cflags without overriding the default CFLAGS. It's not documented and I may be wrong...

[vszakats]

CURL_LDFLAG_EXTRAS_EXE is Makefile.m32 specific, so it indeed doesn't do anything with ./configure builds.

CURL_CFLAG_EXTRAS is also something used only in Makefile.m32. This build method doesn't support a CFLAGS from the environment, so this "extra" envvar is used to inject extra C flags into it.

[bagder]

This issue seems to have stalled. Is there anything we can/should do about this report? Any proposed next steps?

[vszakats]

I think it'd be best to complete Unicode support in libcurl and then fix the different build systems to enable it correctly. Though these two can be done in any order or in parallel. @jay had a nice patch for the former, though I continue to think that fall-back methods are bad and should not be used in libcurl. (Yes, OpenSSL uses one such method, but OpenSSL also keeps at least one Windows vulnerability open since years without intention to fix it (along some other past woes on Windows), so I think it should not be regarded as the yardstick for Windows-development specifically.)

[dEajL3kA]

I think it'd be best to complete Unicode support in libcurl and then fix the different build systems to enable it correctly. Though these two can be done in any order or in parallel. @jay had a nice patch for the former, though I continue to think that fall-back methods are bad and should not be used in libcurl.

👍 ⚡

[bagder]

Anyone care to write this issue up in a brief summary for adding to the KNOWN_BUGS document?

[jay]

IMO this isn't a bug, it's either an enhancement or a feature request. The user wants an easier way to tell autotools to build curl with Windows Unicode support, like ./configure --enable-windows-unicode

[vszakats]

I tried to fix this without the sed trick, and tested: -Dmain=wmain. Oddly enough, it worked!

CPPFLAGS="${CPPFLAGS} -Dmain=wmain"
CPPFLAGS="${CPPFLAGS} -DUNICODE -D_UNICODE"
LDFLAGS="${LDFLAGS} -municode"

It'd be better to restrict -Dmain=wmain internal to the ./configure run, if possible.

If this is a feasible solution, adding an --enable-windows-unicode option is just the icing on the cake.

---

This new warning came up, unrelated to either of the above hacks:

../../src/tool_main.c:239:5: warning: no previous prototype for function 'wmain' [-Wmissing-prototypes]
int wmain(int argc, wchar_t *argv[])
    ^
1 warning generated.

• UPDATE 1: This warning is a known issue and it's explicitly suppressed with CMake.
• UPDATE 2: Fixed #10744