r-minimal
r-minimal copied to clipboard
Published
20 hours ago •
r-hub
r-hub
Fails to build stringfish
Hi everyone,
I'm trying to install the qs package but the build process fails at a dependency of qs called stringfish.
Running:
RUN installr -d \
-t "R-dev file make automake autoconf linux-headers" \
qs
gives me the following message:
--------------------------------------
Installing qs
✔ Updated metadata database: 2.56 MB in 6 files.
ℹ Updating metadata database
✔ Updating metadata database ... done
→ Will install 4 packages.
→ Will download 4 CRAN packages (5.09 MB).
+ RApiSerialize 0.1.0 [bld][cmp][dl] (6.93 kB)
+ RcppParallel 5.1.4 [bld][cmp][dl] (1.97 MB)
+ qs 0.25.1.1 [bld][cmp][dl] (1.96 MB)
+ stringfish 0.15.5 [bld][cmp][dl] (1.16 MB)
ℹ Getting 4 pkgs (5.09 MB)
✔ Got RApiSerialize 0.1.0 (source) (6.93 kB)
✔ Got qs 0.25.1.1 (source) (1.96 MB)
✔ Got RcppParallel 5.1.4 (source) (1.97 MB)
✔ Got stringfish 0.15.5 (source) (1.16 MB)
ℹ Building RApiSerialize 0.1.0
ℹ Building RcppParallel 5.1.4
✔ Built RApiSerialize 0.1.0 (1.2s)
✔ Installed RApiSerialize 0.1.0 (35ms)
✔ Built RcppParallel 5.1.4 (22.2s)
✔ Installed RcppParallel 5.1.4 (50ms)
ℹ Building stringfish 0.15.5
✖ Failed to build stringfish 0.15.5
Error: <callr_remote_error: Failed to build source package 'stringfish'>
in process 46
-->
Failed to build source package 'stringfish', stdout + stderr:
OE> * installing *source* package ‘stringfish’ ...
OE> ** package ‘stringfish’ successfully unpacked and MD5 sums checked
OE> staged installation is only possible with locking
OE> ** using non-staged installation
OE> checking for pkg-config... /usr/bin/pkg-config
OE> stringfish configure script
OE> PCRE2 10.36 dynamic library detected -- skipping PCRE2 compilation
OE> -lpcre2-8
OE>
OE>
OE> configure: creating ./config.status
OE> config.status: creating src/Makevars
OE> ** libs
OE> g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -DPCRE2_CODE_UNIT_WIDTH=8 -DHAVE_CONFIG_H -I. -I'/usr/local/lib/R/library/Rcpp/include' -I'/usr/local/lib/R/library/RcppParallel/include' -I/usr/local/include -fpic -D__MUSL__ -Wall -pedantic -c RcppExports.cpp -o RcppExports.o
OE> g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -DPCRE2_CODE_UNIT_WIDTH=8 -DHAVE_CONFIG_H -I. -I'/usr/local/lib/R/library/Rcpp/include' -I'/usr/local/lib/R/library/RcppParallel/include' -I/usr/local/include -fpic -D__MUSL__ -Wall -pedantic -c sf_functions.cpp -o sf_functions.o
OE> sf_functions.cpp:788:3: error: 'tbb' does not name a type
OE> 788 | tbb::enumerable_thread_specific<iconv_wrapper> iw_latin1;
OE> | ^~~
OE> sf_functions.cpp:789:3: error: 'tbb' does not name a type
OE> 789 | tbb::enumerable_thread_specific<iconv_wrapper> iw_native;
OE> | ^~~
OE> sf_functions.cpp:790:3: error: 'tbb' does not name a type
OE> 790 | tbb::enumerable_thread_specific<sf::pcre2_match_wrapper> pm;
OE> | ^~~
OE> sf_functions.cpp: In constructor 'grepl_worker::grepl_worker(std::string, iconv_wrapper, iconv_wrapper, const sf::pcre2_match_wrapper&, RStringIndexer*, int*)':
OE> sf_functions.cpp:796:31: error: class 'grepl_worker' does not have any field named 'iw_latin1'
OE> 796 | encode_mode(encode_mode), iw_latin1(iw_latin1), iw_native(iw_native), pm(pm), cr(cr), outptr(outptr) {}
OE> | ^~~~~~~~~
OE> sf_functions.cpp:796:53: error: class 'grepl_worker' does not have any field named 'iw_native'
OE> 796 | encode_mode(encode_mode), iw_latin1(iw_latin1), iw_native(iw_native), pm(pm), cr(cr), outptr(outptr) {}
OE> | ^~~~~~~~~
OE> sf_functions.cpp:796:75: error: class 'grepl_worker' does not have any field named 'pm'
OE> 796 | encode_mode(encode_mode), iw_latin1(iw_latin1), iw_native(iw_native), pm(pm), cr(cr), outptr(outptr) {}
OE> | ^~
OE> sf_functions.cpp: In member function 'virtual void grepl_worker::operator()(std::size_t, std::size_t)':
OE> sf_functions.cpp:799:5: error: 'tbb' has not been declared
OE> 799 | tbb::enumerable_thread_specific<sf::pcre2_match_wrapper>::reference pm_local = pm.local();
OE> | ^~~
OE> sf_functions.cpp:799:60: error: expected primary-expression before '>' token
OE> 799 | tbb::enumerable_thread_specific<sf::pcre2_match_wrapper>::reference pm_local = pm.local();
OE> | ^
OE> sf_functions.cpp:799:63: error: '::reference' has not been declared
OE> 799 | tbb::enumerable_thread_specific<sf::pcre2_match_wrapper>::reference pm_local = pm.local();
OE> | ^~~~~~~~~
OE> sf_functions.cpp:800:5: error: 'tbb' has not been declared
OE> 800 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_latin1_local = iw_latin1.local();
OE> | ^~~
OE> sf_functions.cpp:800:50: error: expected primary-expression before '>' token
OE> 800 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_latin1_local = iw_latin1.local();
OE> | ^
OE> sf_functions.cpp:800:53: error: '::reference' has not been declared
OE> 800 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_latin1_local = iw_latin1.local();
OE> | ^~~~~~~~~
OE> sf_functions.cpp:801:5: error: 'tbb' has not been declared
OE> 801 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_native_local = iw_native.local();
OE> | ^~~
OE> sf_functions.cpp:801:50: error: expected primary-expression before '>' token
OE> 801 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_native_local = iw_native.local();
OE> | ^
OE> sf_functions.cpp:801:53: error: '::reference' has not been declared
OE> 801 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_native_local = iw_native.local();
OE> | ^~~~~~~~~
OE> sf_functions.cpp:809:21: error: 'pm_local' was not declared in this scope
OE> 809 | outptr[i] = pm_local.match(q.ptr, q.len);
OE> | ^~~~~~~~
OE> sf_functions.cpp:811:21: error: 'pm_local' was not declared in this scope
OE> 811 | outptr[i] = pm_local.match(q.ptr, q.len);
OE> | ^~~~~~~~
OE> sf_functions.cpp:815:25: error: 'iw_native_local' was not declared in this scope
OE> 815 | auto istr = iw_native_local.convertToString(q.ptr, q.len);
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp:819:27: error: 'pm_local' was not declared in this scope
OE> 819 | outptr[i] = pm_local.match(istr.second.c_str(), istr.second.size());
OE> | ^~~~~~~~
OE> sf_functions.cpp:822:25: error: 'pm_local' was not declared in this scope
OE> 822 | outptr[i] = pm_local.match(q.ptr, q.len);
OE> | ^~~~~~~~
OE> sf_functions.cpp:825:23: error: 'iw_latin1_local' was not declared in this scope
OE> 825 | auto istr = iw_latin1_local.convertToString(q.ptr, q.len);
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp:829:25: error: 'pm_local' was not declared in this scope
OE> 829 | outptr[i] = pm_local.match(istr.second.c_str(), istr.second.size());
OE> | ^~~~~~~~
OE> sf_functions.cpp:832:23: error: 'pm_local' was not declared in this scope
OE> 832 | outptr[i] = pm_local.match(q.ptr, q.len);
OE> | ^~~~~~~~
OE> sf_functions.cpp: At global scope:
OE> sf_functions.cpp:1104:3: error: 'tbb' does not name a type
OE> 1104 | tbb::enumerable_thread_specific<iconv_wrapper> iw_latin1;
OE> | ^~~
OE> sf_functions.cpp:1105:3: error: 'tbb' does not name a type
OE> 1105 | tbb::enumerable_thread_specific<iconv_wrapper> iw_native;
OE> | ^~~
OE> sf_functions.cpp:1106:3: error: 'tbb' does not name a type
OE> 1106 | tbb::enumerable_thread_specific<sf::pcre2_sub_wrapper> ps;
OE> | ^~~
OE> sf_functions.cpp: In constructor 'gsub_worker::gsub_worker(std::string, iconv_wrapper, iconv_wrapper, const sf::pcre2_sub_wrapper&, cetype_t, cetype_t, RStringIndexer*, sf_vec_data&)':
OE> sf_functions.cpp:1114:31: error: class 'gsub_worker' does not have any field named 'iw_latin1'
OE> 1114 | encode_mode(encode_mode), iw_latin1(iw_latin1), iw_native(iw_native), ps(ps), pattern_enc(pattern_enc), replacement_enc(replacement_enc),
OE> | ^~~~~~~~~
OE> sf_functions.cpp:1114:53: error: class 'gsub_worker' does not have any field named 'iw_native'
OE> 1114 | encode_mode(encode_mode), iw_latin1(iw_latin1), iw_native(iw_native), ps(ps), pattern_enc(pattern_enc), replacement_enc(replacement_enc),
OE> | ^~~~~~~~~
OE> sf_functions.cpp:1114:75: error: class 'gsub_worker' does not have any field named 'ps'
OE> 1114 | encode_mode(encode_mode), iw_latin1(iw_latin1), iw_native(iw_native), ps(ps), pattern_enc(pattern_enc), replacement_enc(replacement_enc),
OE> | ^~
OE> sf_functions.cpp: In member function 'virtual void gsub_worker::operator()(std::size_t, std::size_t)':
OE> sf_functions.cpp:1118:5: error: 'tbb' has not been declared
OE> 1118 | tbb::enumerable_thread_specific<sf::pcre2_sub_wrapper>::reference ps_local = ps.local();
OE> | ^~~
OE> sf_functions.cpp:1118:58: error: expected primary-expression before '>' token
OE> 1118 | tbb::enumerable_thread_specific<sf::pcre2_sub_wrapper>::reference ps_local = ps.local();
OE> | ^
OE> sf_functions.cpp:1118:61: error: '::reference' has not been declared
OE> 1118 | tbb::enumerable_thread_specific<sf::pcre2_sub_wrapper>::reference ps_local = ps.local();
OE> | ^~~~~~~~~
OE> sf_functions.cpp:1119:5: error: 'tbb' has not been declared
OE> 1119 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_latin1_local = iw_latin1.local();
OE> | ^~~
OE> sf_functions.cpp:1119:50: error: expected primary-expression before '>' token
OE> 1119 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_latin1_local = iw_latin1.local();
OE> | ^
OE> sf_functions.cpp:1119:53: error: '::reference' has not been declared
OE> 1119 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_latin1_local = iw_latin1.local();
OE> | ^~~~~~~~~
OE> sf_functions.cpp:1120:5: error: 'tbb' has not been declared
OE> 1120 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_native_local = iw_native.local();
OE> | ^~~
OE> sf_functions.cpp:1120:50: error: expected primary-expression before '>' token
OE> 1120 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_native_local = iw_native.local();
OE> | ^
OE> sf_functions.cpp:1120:53: error: '::reference' has not been declared
OE> 1120 | tbb::enumerable_thread_specific<iconv_wrapper>::reference iw_native_local = iw_native.local();
OE> | ^~~~~~~~~
OE> sf_functions.cpp:1129:27: error: 'ps_local' was not declared in this scope; did you mean 'uselocale'?
OE> 1129 | ref[i] = sfstring(ps_local.gsub(q.ptr), CE_UTF8);
OE> | ^~~~~~~~
OE> | uselocale
OE> sf_functions.cpp:1132:27: error: 'ps_local' was not declared in this scope; did you mean 'uselocale'?
OE> 1132 | ref[i] = sfstring(ps_local.gsub(q.ptr), output_enc);
OE> | ^~~~~~~~
OE> | uselocale
OE> sf_functions.cpp:1136:23: error: 'iw_native_local' was not declared in this scope
OE> 1136 | bool iw = iw_native_local.convert(q.ptr, q.len, outstring);
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp:1140:33: error: 'ps_local' was not declared in this scope; did you mean 'uselocale'?
OE> 1140 | ref[i] = sfstring(ps_local.gsub(outstring.c_str()), CE_UTF8);
OE> | ^~~~~~~~
OE> | uselocale
OE> sf_functions.cpp:1143:31: error: 'ps_local' was not declared in this scope; did you mean 'uselocale'?
OE> 1143 | ref[i] = sfstring(ps_local.gsub(q.ptr), CE_UTF8);
OE> | ^~~~~~~~
OE> | uselocale
OE> sf_functions.cpp:1146:21: error: 'iw_latin1_local' was not declared in this scope
OE> 1146 | bool iw = iw_latin1_local.convert(q.ptr, q.len, outstring);
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp:1150:31: error: 'ps_local' was not declared in this scope; did you mean 'uselocale'?
OE> 1150 | ref[i] = sfstring(ps_local.gsub(outstring.c_str()), CE_UTF8);
OE> | ^~~~~~~~
OE> | uselocale
OE> sf_functions.cpp:1153:29: error: 'ps_local' was not declared in this scope; did you mean 'uselocale'?
OE> 1153 | ref[i] = sfstring(ps_local.gsub(q.ptr), CE_UTF8);
OE> | ^~~~~~~~
OE> | uselocale
OE> sf_functions.cpp: At global scope:
OE> sf_functions.cpp:1348:25: error: 'tbb' does not name a type
OE> 1348 | using tbb_rstring_map = tbb::concurrent_unordered_map<RStringIndexer::rstring_info, tbb::atomic<int>, rstring_info_hash>;
OE> | ^~~
OE> sf_functions.cpp:1352:3: error: 'tbb_rstring_map' does not name a type
OE> 1352 | tbb_rstring_map * table_hash;
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp:1354:35: error: expected ')' before '*' token
OE> 1354 | hash_fill_worker(tbb_rstring_map * t, RStringIndexer * f) :
OE> | ~ ^~
OE> | )
OE> sf_functions.cpp: In member function 'virtual void hash_fill_worker::operator()(std::size_t, std::size_t)':
OE> sf_functions.cpp:1360:16: error: 'table_hash' was not declared in this scope
OE> 1360 | auto p = table_hash->insert(tbb_rstring_map::value_type(q,i));
OE> | ^~~~~~~~~~
OE> sf_functions.cpp:1360:35: error: 'tbb_rstring_map' has not been declared
OE> 1360 | auto p = table_hash->insert(tbb_rstring_map::value_type(q,i));
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp: At global scope:
OE> sf_functions.cpp:1373:3: error: 'tbb_rstring_map' does not name a type
OE> 1373 | tbb_rstring_map * table_hash;
OE> | ^~~~~~~~~~~~~~~
OE> sf_functions.cpp:1376:37: error: expected ')' before '*' token
OE> 1376 | hash_search_worker(tbb_rstring_map * t, RStringIndexer * s, int * o) :
OE> | ~ ^~
OE> | )
OE> sf_functions.cpp: In member function 'virtual void hash_search_worker::operator()(std::size_t, std::size_t)':
OE> sf_functions.cpp:1382:17: error: 'table_hash' was not declared in this scope
OE> 1382 | auto it = table_hash->find(q);
OE> | ^~~~~~~~~~
OE> make: *** [/usr/local/lib/R/etc/Makeconf:177: sf_functions.o] Error 1
OE> ERROR: compilation failed for package ‘stringfish’
OE> * removing ‘/tmp/RtmpClEiCf/pkg-lib2e5be10640/stringfish’
Stack trace:
12. (function (...) ...
13. base:::withCallingHandlers(cli_message = function(msg) { ...
14. get("pkg_install_do_plan", asNamespace("pak"))(...)
15. pkgdepends::install_package_plan(plan = plan, lib = lib, num_workers = num_ ...
16. base:::withCallingHandlers({ ...
17. pkgdepends:::handle_events(state, events)
18. pkgdepends:::handle_event(state, i)
19. pkgdepends:::stop_task(state, worker)
20. pkgdepends:::stop_task_build(state, worker)
21. base:::throw(new_pkg_build_error("Failed to build source package {pkg}", ...
22. base:::signalCondition(cond)
23. (function (e) ...
24. base:::stop(e)
25. (function (e) ...
x Failed to build source package 'stringfish'
Execution halted
--------------------------------------
I'm not too familiar with the messages I'm seeing but from what I understand it might have something to do with locales and time zone data?
So I tried adding tzdata without success.
RUN apk add --no-cache tzdata
RUN export TZDIR=/usr/share/zoneinfo
RUN installr -d \
-t "R-dev file make automake autoconf linux-headers" \
qs
Any ideas or pointers? Thanks in advance.
I tried some obvious things, but got nowhere. Maybe it is worth reporting this for stringfish, because it fails the same way in a vanilla Alpine container:
docker run -ti alpine:latest sh
apk add R gcc g++ R-dev make linux-headers
R -q -e 'options(repos = "https://cloud.r-project.org"); install.packages("stringfish")'
I'm not familiar with alpine linux, but the error message (from the issue: https://github.com/traversc/stringfish/issues/11) suggests something with the tbb source files from RcppParallel:
../../src/tbbmalloc/proxy.cpp:27:47: error: missing binary operator before token "("
27 | #if defined(__GLIBC_PREREQ) && !__GLIBC_PREREQ(2, 16) && _GLIBCXX_HAVE_ALIGNED_ALLOC
| ^
../../src/tbbmalloc/proxy.cpp:276:26: error: return type 'struct mallinfo' is incomplete
276 | struct mallinfo mallinfo() __THROW
| ^
... (etc)
So my suspicion is that there's an issue there. It might make sense to try to do a minimal package using tbb/RcppParallel to confirm if that is the case.
Maybe TBB is not great on musl? https://patchwork.openembedded.org/patch/171846/
I think it's likely, but I think a bit more work is needed to confirm.
What is musl/alpine?
Is there a macro #define that one can check?
There is no official #define AFAIK, but in this container we do set __MUSL__ in CFLAGS and CXXFLAGS.
~~I have identified the issue and it's indeed a RcppParallel issue, but thankfully not a serious one.~~
Edit: There are two issues:
- A missing macro issue in Alpine, which should be resolved by RcppParallel
- An issue on my end in that I'm not properly checking for TBB (should be fixed on github soon)
Below describes the missing macro issue:
RcppParallel has this macro define:
#ifndef RCPP_PARALLEL_USE_TBB
# if defined(__APPLE__) || defined(__gnu_linux__) || (defined(__sun) && defined(__SVR4) && !defined(__sparc))
# define RCPP_PARALLEL_USE_TBB 1
# else
# define RCPP_PARALLEL_USE_TBB 0
# endif
#endif
On ubuntu, RCPP_PARALLEL_USE_TBB should usually be set to 1 because of __gnu_linux__, but on Alpine this macro isn't set.
The reason this isn't set on Alpine is the __gnu_linux__ macro is defined in glibc but not in musl: https://stackoverflow.com/questions/66037729/what-does-gnu-linux-stand-for
Here's a script to test these macros:
library(Rcpp)
sourceCpp(code = '
// [[Rcpp::depends(RcppParallel)]]
#include <RcppParallel.h>
#include <Rcpp.h>
// [[Rcpp::export]]
void test_macros() {
std::cout << "RCPP_PARALLEL_USE_TBB = " << RCPP_PARALLEL_USE_TBB << std::endl;
#ifdef __APPLE__
std::cout << "__APPLE__" << std::endl;
#endif
#ifdef __gnu_linux__
std::cout << "__gnu_linux__" << std::endl;
#endif
#ifdef __sun
std::cout << "__sun" << std::endl;
#endif
#ifdef __SVR4
std::cout << "__SVR4" << std::endl;
#endif
#ifdef __sparc
std::cout << "__sparc" << std::endl;
#endif
}
')
Here's a MWE that demonstrates the issue (compiles on Ubuntu, but not on Alpine).
library(Rcpp)
sourceCpp(code = '
// [[Rcpp::depends(RcppParallel)]]
#include <RcppParallel.h>
#include <Rcpp.h>
#include <cstdlib>
#if RCPP_PARALLEL_USE_TBB
#include <tbb/enumerable_thread_specific.h>
#endif
using namespace RcppParallel;
using namespace Rcpp;
struct TestWorker : public Worker {
RVector<int> output;
tbb::enumerable_thread_specific<std::vector<int>> thread_id;
TestWorker(IntegerVector output) : output(output) {}
void operator()(std::size_t begin, std::size_t end) {
for(size_t i=begin; i<end; ++i) {
thread_id.local().push_back(i);
output[i] = thread_id.local().size();
}
}
};
// [[Rcpp::export]]
IntegerVector test(size_t n) {
IntegerVector output(n);
TestWorker t(output);
parallelFor(0, n, t, 1,2);
return output;
}
')
Presumably there is a similar macro that could be checked on Alpine. It might be a good idea to ask K. Ushey's opinion, I'll leave that to you.
I believe __linux__ should be set on all Linux systems, if the goal is to detect Linux.
TBB seems to work on Alpine if you force it. Maybe there is a reason they chose to check __gnu_linux__ instead.
A year or so later, this does not seem to be a problem any more, and stringfish installs cleanly.