jeroen
Windows ARM64 source build - fails if parallel make jobs used (winlibs download race condition)
Currently experiencing this on Windows ARM64 (so not sure if present in x86) with all packages that need winlibs static libs (where the library isn't already in rtools).
If I have MAKEFLAGS=-j2 (or any value > 1) set in my .Renviron, the installation will fail at the linking stage due to libraries not being found:
install.packages("curl", repos = "https://cloud.r-project.org")
...
"C:/PROGRA~1/R-AARC~1/R-devel/bin/Rscript.exe" "../tools/winlibs.R"
clang -shared -s -static-libgcc -o curl.dll tmp.def callbacks.o curl.o download.o escape.o fetch.o findport.o form.o getdate.o handle.o ieproxy.o init.o interrupt.o multi.o nslookup.o options.o reflist.o split.o ssl.o typechecking.o utils.o version.o winidn.o writer.o -L../windows/libcurl/libclang-17.0.6 -L../windows/libcurl/lib -lwinhttp -lcurl -lnghttp2 -lssh2 -lz -lssl -lcrypto -pthread -lgdi32 -lws2_32 -lcrypt32 -lbcrypt -lwldap32 -L/aarch64-w64-mingw32.static.posix/lib -L/aarch64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R-AARC~1/R-devel/bin -lR
lld: error: unable to find library -lnghttp2
clang-17: error: linker command failed with exit code 1 (use -v to see invocation)
[1] TRUE
echo '#include <curl/curl.h>' | clang -I../windows/libcurl/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -DR_NO_REMAP -std=gnu99 -E -xc - | grep "^[ \t]*CURLOPT_.*," | sed s/,// > ../tools/option_table.txt
no DLL was created
ERROR: compilation failed for package 'curl'
Which appears to be caused by $(SHLIB) target reaching the linking stage before the winlibs target has completed. I'd guess that this hasn't been seen to-date on x86_64 because compilation is generally slower than under ARM, so the winlibs target has usually finished before $(SHLIB) reaches linking.
From local testing, an easy fix appears to be just making the dependency explicit in Makevars.win:
- all: clean winlibs
+ $(SHLIB): $(OBJECTS)
+ $(OBJECTS): winlibs
But I'm not sure whether there's an easier fix which doesn't require updating all winlibs R packages. Thanks!
Yes I think that is the only solution.
FWIW this is unrelated to arm but about using parallel make. Afaik parallel make on Windows was never really supported because there are many other problems with it, so we didn't bother to test it.
I'm sorry I tried it but it just broke the build, so I hard to revert. The Windows tooling is a mess these days, I don't care enough to fix this. Just don't use parallel make.