adcomp icon indicating copy to clipboard operation
adcomp copied to clipboard

Published 20 hours ago • verified publisher icon kaskr

Some models (e.g. VAST) fail to compile with newer compilers

Open jkbest2 opened this issue 5 years ago • 17 comments

Issue

I am having trouble getting newer versions of VAST to compile. Others are encountering similar difficulties (e.g. James-Thorson-NOAA/VAST#140). In that issue Jim suggested posting here for more feedback on the compiler errors. I have successfully compiled and fit other models using the same versions of TMB and compilers.

Reproducible example

To reproduce the issue, run

library(TMB)

download.file("https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v8_0_0.cpp", "VAST_v8_0_0.cpp")

compile("VAST_v8_0_0.cpp")

The output from clang++ and g++ available in this Gist, but the three errors from clang++ are

In file included from VAST_v8_0_0.cpp:1:
In file included from /home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/TMB/include/TMB.hpp:53:
In file included from /home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/Eigen/Dense:1:
In file included from /home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/Eigen/Core:371:
/home/jkbest/R/x86_64-pc-linux-gnu-library/3.6/RcppEigen/include/Eigen/src/Core/MathFunctions.h:713:12: error: no matching function for call to 'isfinite'
    return isfinite EIGEN_NOT_A_MACRO (x);
^~~~~~~~

The errors output by g++ are similar, but more verbose.

Version information

OS: Kubuntu Linux 19.04

R: 3.6.0 TMB: 1.7.15 RcppEigen: 0.3.3.5.0

clang++: 8.0.0-3 g++: 8.3.0

I get the same results using clang++ 6 and g++ 6. These are the earliest versions easily available for my distribution.

jkbest2 avatar Jun 11 '19 20:06 jkbest2

@jkbest2 Should be fixed - please test.

kaskr avatar Jun 12 '19 08:06 kaskr

Thank you!

VAST now compiles for me using both clang++ 8 and gcc 8. This appears to be resolved.

jkbest2 avatar Jun 12 '19 14:06 jkbest2

testing on a mac, did fresh install with original mac clang and clang7 (recommended for R vers 3.6) and still get locked file error on test: r$> devtools::install_github("kaskr/adcomp/TMB")

Downloading GitHub repo kaskr/adcomp@master checking for file ‘/private/var/folders/qb/mr3sb1m1403cqwg1yy0qtjz80000gn/T/RtmpHqATeT/remotesf324121c429/kaskr-adcomp-acf0140/TMB/DESCRIPT✔ checking for file ‘/private/var/folders/qb/mr3sb1m1403cqwg1yy0qtjz80000gn/T/RtmpHqATeT/remotesf324121c429/kaskr-adcomp-acf0140/TMB/DESCRIPTION’ ─ preparing ‘TMB’: (1.9s) ✔ checking DESCRIPTION meta-information ─ cleaning src ─ checking for LF line-endings in source and make files and shell scripts ─ checking for empty or unneeded directories (1.7s) ─ building ‘TMB_1.7.15.tar.gz’

  • installing source package ‘TMB’ ... ** using staged installation ** libs clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Matrix/include" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c external_metis.c -o external_metis.o clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Matrix/include" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c init.c -o init.o clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Matrix/include" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c local_stubs.c -o local_stubs.o clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Matrix/include" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c solve_subset.c -o solve_subset.o clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Matrix/include" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c utils.c -o utils.o clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o TMB.so external_metis.o init.o local_stubs.o solve_subset.o utils.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation installing to /Library/Frameworks/R.framework/Versions/3.6/Resources/library/00LOCK-TMB/00new/TMB/libs ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices ** building package indices ** testing if installed package can be loaded from temporary location

Error: package or namespace load failed for ‘TMB’: .onLoad failed in loadNamespace() for 'TMB', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/Library/Frameworks/R.framework/Versions/3.6/Resources/library/00LOCK-TMB/00new/TMB/libs/TMB.so': dlopen(/Library/Frameworks/R.framework/Versions/3.6/Resources/library/00LOCK-TMB/00new/TMB/libs/TMB.so, 6): Symbol not found: ___addtf3 Referenced from: /usr/local/lib/libquadmath.0.dylib Expected in: /usr/local/lib/libgcc_s_x86_64.1.dylib in /usr/local/lib/libquadmath.0.dylib Error: loading failed Execution halted ERROR: loading failed

  • removing ‘/Library/Frameworks/R.framework/Versions/3.6/Resources/library/TMB’ Error in i.p(...) : (converted from warning) installation of package ‘/var/folders/qb/mr3sb1m1403cqwg1yy0qtjz80000gn/T//RtmpHqATeT/filef3248eb2855/TMB_1.7.15.tar.gz’ had non-zero exit status

On Wed, Jun 12, 2019 at 7:05 AM John Best [email protected] wrote:

Thank you!

VAST now compiles for me using both clang++ 8 or gcc 8. This appears to be resolved.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAUW7YVK3SFG6LZ3HNRKLHDP2D7DRA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXQRBAQ#issuecomment-501289090, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUW7YTU7CR4HPEJPKJT4QTP2D7DRANCNFSM4HXC5URA .

-- Jim Ianelli

jimianelli avatar Jun 12 '19 22:06 jimianelli

@jimianelli This is unlikely a TMB issue. FWIW, the message Symbol not found: ___addtf3* indicates a linkage problem on your Mac system. The library /usr/local/lib/libquadmath.0.dylib seems broken?

kaskr avatar Jun 13 '19 07:06 kaskr

Edit: Sorry, I see you already installed the correct clang version. Maybe it's a PATH issue and the correct one isn't always being called?

I suspect @kaskr is correct. It looks like compilers on Macs can be a bit finicky, but the required compilers for your OS X and R versions should be available here.

jkbest2 avatar Jun 13 '19 16:06 jkbest2

thanks, yes, realized it wasn't a TMB issue but hoping that others experienced a similar problem and had a fix. Thanks John and yes, that was the clean (?) install I used (R 3.6 with clang7).

On Thu, Jun 13, 2019 at 9:01 AM John Best [email protected] wrote:

I suspect @kaskr https://github.com/kaskr is correct. It looks like compilers on Macs can be a bit finicky https://groups.google.com/forum/#!msg/r-sig-mac/qcGdcfnSTsc/hks1T5eKBwAJ, but the required compilers for your OS X and R versions should be available here https://cran.r-project.org/bin/macosx/tools/.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAUW7YW7IRBBUFT2VK72W5DP2JVNJA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXUFTEI#issuecomment-501766545, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUW7YQHEQXYLBSI6JTLZUDP2JVNJANCNFSM4HXC5URA .

-- Jim Ianelli

jimianelli avatar Jun 13 '19 18:06 jimianelli

I find a solution for TMB (R 3.6, MacOS Mojave 10.14.5), which was:

In the terminal:

brew install llvm

After the make a file: ~/.R/Makevars with

CC=/usr/local/opt/llvm/bin/clang
CXX=/usr/local/opt/llvm/bin/clang++

After that, to install TMB

install.packages("TMB",type="source")

TMB is fine with numerous examples, but the installation has trouble and was not able to compile

compile("VAST_v8_0_0.cpp")

Prof. Dr. Luis Cubillos Profesor Asociado Departamento Oceanografía Facultad de Ciencias Naturales y Oceanográficas Tel.: 56-41-2 207 233 Email: [email protected] Scholar Google Citations http://scholar.google.es/citations?hl=es&user=pLQd6tQAAAAJ ResearchGate https://www.researchgate.net/profile/Luis_Cubillos

El jue., 13 jun. 2019 a las 14:28, Jim Ianelli ([email protected]) escribió:

thanks, yes, realized it wasn't a TMB issue but hoping that others experienced a similar problem and had a fix. Thanks John and yes, that was the clean (?) install I used (R 3.6 with clang7).

On Thu, Jun 13, 2019 at 9:01 AM John Best [email protected] wrote:

I suspect @kaskr https://github.com/kaskr is correct. It looks like compilers on Macs can be a bit finicky < https://groups.google.com/forum/#!msg/r-sig-mac/qcGdcfnSTsc/hks1T5eKBwAJ>, but the required compilers for your OS X and R versions should be available here https://cran.r-project.org/bin/macosx/tools/.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAUW7YW7IRBBUFT2VK72W5DP2JVNJA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXUFTEI#issuecomment-501766545 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAUW7YQHEQXYLBSI6JTLZUDP2JVNJANCNFSM4HXC5URA

.

-- Jim Ianelli

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAP3OPHOXJFITV6LS74EHVTP2KGWRA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXUUHGA#issuecomment-501826456, or mute the thread https://github.com/notifications/unsubscribe-auth/AAP3OPB6DH4RMYRTE7SPF7TP2KGWRANCNFSM4HXC5URA .

LuisACubillos avatar Jun 13 '19 20:06 LuisACubillos

@LuisACubillos So, you get the VAST compilation error when using the current CRAN version of TMB. This is as expected. Would you mind testing if devtools::install_github("kaskr/adcomp/TMB") solves the problem?

kaskr avatar Jun 13 '19 20:06 kaskr

I re-installed TMB accordingly (i.e., devtools::install_github(" kaskr/adcomp/TMB"), and I tested with simple and other examples. It was fine. However, I tested the following:

library(TMB)

download.file("https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v8_0_0.cpp", "VAST_v8_0_0.cpp")

Version="VAST_v8_0_0" compile( paste0(Version,".cpp") )

After some verbose on the console, it was not able to compile, i.e.,

5 warnings and 9 errors generated. make: *** [VAST_v8_0_0.o] Error 1 Error in compile(paste0(Version, ".cpp")) : Compilation failed

Luis

El jue., 13 jun. 2019 a las 16:49, kaskr ([email protected]) escribió:

@LuisACubillos https://github.com/LuisACubillos So, you get the VAST compilation error when using the current CRAN version of TMB. This is as expected. Would you mind testing if devtools::install_github("kaskr/adcomp/TMB") solves the problem?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAP3OPGURMYRFMIEHPLZFNTP2KXFHA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXU7V5I#issuecomment-501873397, or mute the thread https://github.com/notifications/unsubscribe-auth/AAP3OPGM2EQFY2DGS62DJRTP2KXFHANCNFSM4HXC5URA .

LuisACubillos avatar Jun 14 '19 00:06 LuisACubillos

@LuisACubillos OK thanks for testing - it seems this issue is far from solved.

kaskr avatar Jun 14 '19 08:06 kaskr

I resolved the libquadmath issue by setting the ~/.R/Makevars to contain: CC=/usr/local/clang7/bin/clang CXX=/usr/local/clang7/bin/clang++ CXX11=$CXX CXX14=$CXX CXX17=$CXX CXX1X=$CXX LDFLAGS=-L/usr/local/clang7/lib

where clang7 is the recommended "tools" for macos R 3.6.

However, library(TMB) download.file(" https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v5_0_0.cpp", "VAST_v5_0_0.cpp") compile("VAST_v5_0_0.cpp") still fails with "error: no matching function for call to 'scalbn' yet as Curry Cunningham found, compiling VAST version 4 works... download.file(" https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v4_0_0.cpp", "VAST_v4_0_0.cpp") compile("VAST_v4_0_0.cpp")

Next I will try clang 8.0 because I believe that's what John Best was using for linux version...

On Thu, Jun 13, 2019 at 5:49 PM Luis A Cubillos [email protected] wrote:

I re-installed TMB accordingly (i.e., devtools::install_github(" kaskr/adcomp/TMB"), and I tested with simple and other examples. It was fine. However, I tested the following:

library(TMB)

download.file(" https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v8_0_0.cpp ", "VAST_v8_0_0.cpp")

Version="VAST_v8_0_0" compile( paste0(Version,".cpp") )

After some verbose on the console, it was not able to compile, i.e.,

5 warnings and 9 errors generated. make: *** [VAST_v8_0_0.o] Error 1 Error in compile(paste0(Version, ".cpp")) : Compilation failed

Luis

El jue., 13 jun. 2019 a las 16:49, kaskr ([email protected]) escribió:

@LuisACubillos https://github.com/LuisACubillos So, you get the VAST compilation error when using the current CRAN version of TMB. This is as expected. Would you mind testing if devtools::install_github("kaskr/adcomp/TMB") solves the problem?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAP3OPGURMYRFMIEHPLZFNTP2KXFHA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXU7V5I#issuecomment-501873397 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAP3OPGM2EQFY2DGS62DJRTP2KXFHANCNFSM4HXC5URA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAUW7YQVLGZIIAE55QJ4JWDP2LTLDA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXVNEIQ#issuecomment-501928482, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUW7YRCPBYQK2Y32OON2F3P2LTLDANCNFSM4HXC5URA .

-- Jim Ianelli

jimianelli avatar Jun 14 '19 17:06 jimianelli

Jim, Did you delete anything in your makevars file? Or end up getting things working.

Removed my install of R 3.3.3 and can’t seem to get TMB working again on Mac.

Thanks

On Fri, Jun 14, 2019 at 7:57 PM Jim Ianelli [email protected] wrote:

I resolved the libquadmath issue by setting the ~/.R/Makevars to contain: CC=/usr/local/clang7/bin/clang CXX=/usr/local/clang7/bin/clang++ CXX11=$CXX CXX14=$CXX CXX17=$CXX CXX1X=$CXX LDFLAGS=-L/usr/local/clang7/lib

where clang7 is the recommended "tools" for macos R 3.6.

However, library(TMB) download.file("

https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v5_0_0.cpp ", "VAST_v5_0_0.cpp") compile("VAST_v5_0_0.cpp") still fails with "error: no matching function for call to 'scalbn' yet as Curry Cunningham found, compiling VAST version 4 works... download.file("

https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v4_0_0.cpp ", "VAST_v4_0_0.cpp") compile("VAST_v4_0_0.cpp")

Next I will try clang 8.0 because I believe that's what John Best was using for linux version...

On Thu, Jun 13, 2019 at 5:49 PM Luis A Cubillos [email protected] wrote:

I re-installed TMB accordingly (i.e., devtools::install_github(" kaskr/adcomp/TMB"), and I tested with simple and other examples. It was fine. However, I tested the following:

library(TMB)

download.file("

https://raw.githubusercontent.com/James-Thorson-NOAA/VAST/master/inst/executables/VAST_v8_0_0.cpp

", "VAST_v8_0_0.cpp")

Version="VAST_v8_0_0" compile( paste0(Version,".cpp") )

After some verbose on the console, it was not able to compile, i.e.,

5 warnings and 9 errors generated. make: *** [VAST_v8_0_0.o] Error 1 Error in compile(paste0(Version, ".cpp")) : Compilation failed

Luis

El jue., 13 jun. 2019 a las 16:49, kaskr ([email protected]) escribió:

@LuisACubillos https://github.com/LuisACubillos So, you get the VAST compilation error when using the current CRAN version of TMB. This is as expected. Would you mind testing if devtools::install_github("kaskr/adcomp/TMB") solves the problem?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAP3OPGURMYRFMIEHPLZFNTP2KXFHA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXU7V5I#issuecomment-501873397

, or mute the thread <

https://github.com/notifications/unsubscribe-auth/AAP3OPGM2EQFY2DGS62DJRTP2KXFHANCNFSM4HXC5URA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAUW7YQVLGZIIAE55QJ4JWDP2LTLDA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXVNEIQ#issuecomment-501928482 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AAUW7YRCPBYQK2Y32OON2F3P2LTLDANCNFSM4HXC5URA

.

-- Jim Ianelli

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AGEPVQSKKILZ74GIFOB5YOTP2PLWXA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXXQ4JY#issuecomment-502205991, or mute the thread https://github.com/notifications/unsubscribe-auth/AGEPVQRSTDUWERHFHQZNXLDP2PLWXANCNFSM4HXC5URA .

-- Grant Adams School of Aquatic Fishery Sciences University of Washington https://grantdadams.wordpress.com/

grantdadams avatar Jun 29 '19 08:06 grantdadams

If someone has a better solution for macOS, please provide an update.

I had the same problem described above on macOS, but used this workaround to run VAST; hope this helps. TMB installed fine with clang6, and examples worked both from CRAN and using devtools::install_github("kaskr/adcomp/TMB"), but VAST compilation failed for v 8.2. (I was able to get VAST v 4 to work as Jim described above.)

However, I changed my ~/.R/Makevars after installing TMB and VAST to the following, and now VAST v 8.2 can run the simple example here.

##CC=/usr/local/clang6/bin/clang ##CXX=/usr/local/clang6/bin/clang++ CC=gcc CXX=/usr/local/bin/g++-9 CXX1X=/usr/local/clang6/bin/clang++ CXX98=/usr/local/clang6/bin/clang++ CXX11=/usr/local/clang6/bin/clang++ CXX14=/usr/local/clang6/bin/clang++ CXX17=/usr/local/clang6/bin/clang++ LDFLAGS=-L/usr/local/clang6/lib

My configuration:

NECLWHSGAICHAS-2: sgaichas$ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.0.0 (clang-900.0.39.2) Target: x86_64-apple-darwin16.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

sessionInfo() R version 3.5.2 (2018-12-20) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Sierra 10.12.6

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] FishStatsUtils_2.2.0 mapdata_2.3.0 maps_3.3.0 VAST_3.1.0
[5] TMB_1.7.15

loaded via a namespace (and not attached): [1] RANN_2.6.1 lattice_0.20-38 TMBhelper_1.1.0 grid_3.5.2
[5] ThorsonUtilities_1.0 INLA_18.07.12 sp_1.3-1 Matrix_1.2-16
[9] splines_3.5.2 tools_3.5.2 splancs_2.01-40 compiler_3.5.2
[13] PBSmapping_2.72.1

sgaichas avatar Aug 06 '19 14:08 sgaichas

Can confirm that Sarah's solution here for installing TMB and successfully compiling VAST 8 also worked for me using R 3.6.1 and clang7. (thanks Sarah!)

gavinfay avatar Aug 19 '19 12:08 gavinfay

I have a similar trouble when compiling VAST_8_5_0.cpp from the R command

TMB::compile("VAST_v8_5_0.cpp")

On my Mac (MacOS 10.15.3) with Apple clang++

$ clang++ -v
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin19.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

and R-3.6.2 installed by source by homebrew.

On the other hand , when I used Sarah's approach to specify

CXX=/usr/local/bin/g++-9

in ~/.R/Makevars to use g++ compiler installed from source by homebrew,

TMB::compile("VAST_v8_5_0.cpp")

finished without error.

More generally I found the 'TMB::compile("VAST_v***.cpp")' failed for VAST_v5_0_0.cpp and newer ones if Apple's clang++ compiler is used.

Particular differences between VAST_v5_0_0.cpp and newer ones and previous VAST_v*.cpp is
the use of "std::complex<Type>"

vector< std::complex<Type> > eigenvalues_B_ff = B_ff.eigenvalues();

In fact compilation errors, when clang++ is used to compile VAST_v8_5_0.cpp, comes from

  1. Error related to Implicit cast from int to double or float eg.
VAST_v8_5_0.cpp:688:18: error: call to 'sqrt' is ambiguous
    Range_raw1 = sqrt(8) / exp( logkappa1 );   // Range = approx. distance @ 10% correlation

This error is very common for clang(++)

  1. Errors probably related to instantiation of std::complex<> Eg.
In file included from VAST_v8_5_0.cpp:1:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/TMB.hpp:53:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Dense:1:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Core:80:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:677:24: error: no matching function for call to 'fmax'
    _Tp __logbw = logb(fmax(fabs(__c), fabs(__d)));
                       ^~~~
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/EigenSolver.h:542:74: note: in instantiation of function template specialization
      'std::__1::operator/<CppAD::AD<double> >' requested here
        ComplexScalar cc = ComplexScalar(Scalar(0),-m_matT.coeff(n-1,n)) / ComplexScalar(m_matT.coeff(n-1,n-1)-p,q);
                                                                         ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/EigenSolver.h:447:7: note: in instantiation of member function
      'Eigen::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::doComputeEigenvectors' requested here
      doComputeEigenvectors();
      ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/EigenSolver.h:156:7: note: in instantiation of function template specialization
      'Eigen::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::compute<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >' requested here
      compute(matrix.derived(), computeEigenvectors);
      ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/MatrixBaseEigenvalues.h:39:12: note: in instantiation of function template specialization
      'Eigen::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::EigenSolver<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >' requested here
    return EigenSolver<PlainObject>(m_eval, false).eigenvalues();
           ^
/usr/local/lib/R/3.6/site-library/TMB/include/Eigen/src/Eigenvalues/MatrixBaseEigenvalues.h:70:81: note: in instantiation of member function
      'Eigen::internal::eigenvalues_selector<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1>, false>::run' requested here
  return internal::eigenvalues_selector<Derived, NumTraits<Scalar>::IsComplex>::run(derived());
                                                                                ^
VAST_v8_5_0.cpp:439:58: note: in instantiation of member function 'Eigen::MatrixBase<Eigen::Matrix<CppAD::AD<double>, -1, -1, 0, -1, -1> >::eigenvalues' requested here
    vector< std::complex<Type> > eigenvalues_B_ff = B_ff.eigenvalues();
                                                         ^
VAST_v8_5_0.cpp:727:10: note: in instantiation of function template specialization 'calculate_B<CppAD::AD<double> >' requested here
  B_ff = calculate_B( VamConfig(0), n_f1, VamConfig(1), Chi_fr, Psi_fr, jnll_comp(13) );
         ^
/usr/local/lib/R/3.6/site-library/TMB/include/tmb_core.hpp:1135:6: note: in instantiation of member function 'objective_function<CppAD::AD<double> >::operator()'
      requested here
    F(); // Run through user template (modifies reportvector)
     ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/math.h:532:15: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'double' for 1st argument
extern double fmax(double, double);
              ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1217:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'float' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY float       fmax(float __lcpp_x, float __lcpp_y) _NOEXCEPT             {return ::fmaxf(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1218:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'long double' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY long double fmax(long double __lcpp_x, long double __lcpp_y) _NOEXCEPT {return ::fmaxl(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1228:1: note: candidate template ignored: substitution
      failure [with _A1 = CppAD::AD<double>, _A2 = CppAD::AD<double>]: no type named 'type' in 'std::__1::__lazy_enable_if<false,
      std::__1::__promote<CppAD::AD<double>, CppAD::AD<double>, void> >'
fmax(_A1 __lcpp_x, _A2 __lcpp_y) _NOEXCEPT
^

First one is easy to solve by changing sqrt(8) to sqrt(8.0) or by adding a trivial new function

double sqrt(int x){
  return sqrt(double(x));
}

As for second one, probably similar problem was reported here and posted [their fix] (https://github.com/su2code/SU2/pull/351) for their library. In essence, they claimed that "complex" header from llvm uses some function specifically assumes their arguments to be double. They caused error when std::complexCppAD::AD was instantiated.

I confirmed, similar error related to std:comlex<Type> happens, when I try compiling more simple artificial code

// Simple linear regression.
#include <TMB.hpp>
template<class Type>
Type objective_function<Type>::operator() ()
{
  DATA_VECTOR(Y);
  DATA_VECTOR(x);
  PARAMETER(a);
  PARAMETER(b);
  PARAMETER(logSigma);
  std::complex<Type> c1(3.0, 4.0);
  std::complex<Type> c2(1.0, 1.56);
  std::complex<Type> c3=c1 / c2;
  ADREPORT(exp(2*logSigma));
  Type nll = -sum(dnorm(Y, a+b*x, exp(logSigma), true));
  Rcout << "nll : " << std::endl;
  return nll;
}

Similar error was produced and its compilation was failed

In file included from test.cpp:2:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/TMB.hpp:53:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Dense:1:
In file included from /usr/local/lib/R/3.6/site-library/TMB/include/Eigen/Core:80:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:677:24: error: no matching function for call to 'fmax'
    _Tp __logbw = logb(fmax(fabs(__c), fabs(__d)));
                       ^~~~
test.cpp:13:28: note: in instantiation of function template specialization 'std::__1::operator/<CppAD::AD<double> >' requested here
  std::complex<Type> c3=c1 / c2;
                           ^
/usr/local/lib/R/3.6/site-library/TMB/include/tmb_core.hpp:1135:6: note: in instantiation of member function 'objective_function<CppAD::AD<double> >::operator()'
      requested here
    F(); // Run through user template (modifies reportvector)
     ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/math.h:532:15: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'double' for 1st argument
extern double fmax(double, double);
              ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1217:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'float' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY float       fmax(float __lcpp_x, float __lcpp_y) _NOEXCEPT             {return ::fmaxf(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1218:46: note: candidate function not viable: no known
      conversion from 'AD<double>' to 'long double' for 1st argument
inline _LIBCPP_INLINE_VISIBILITY long double fmax(long double __lcpp_x, long double __lcpp_y) _NOEXCEPT {return ::fmaxl(__lcpp_x, __lcpp_y);}
                                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:1228:1: note: candidate template ignored: substitution
      failure [with _A1 = CppAD::AD<double>, _A2 = CppAD::AD<double>]: no type named 'type' in 'std::__1::__lazy_enable_if<false,
      std::__1::__promote<CppAD::AD<double>, CppAD::AD<double>, void> >'
fmax(_A1 __lcpp_x, _A2 __lcpp_y) _NOEXCEPT

In addition if I put a line CXX=g++-9 in ~/.R/Makevars TMB::compile command finished without any error.

yukio-takeuchi avatar Feb 12 '20 02:02 yukio-takeuchi

@yukio-takeuchi The problem is not limited to Mac. I can replicate the issue on a Linux system using any version of clang up to 11.0.0. (However, the LLVM version of the C++ standard library -stdlib has to be explicitely selected):

TMB:::compile("VAST_v8_0_0.cpp", CXX="clang++ -stdlib=libc++ -std=c++98")

To 'fix' the compilation I switch to the gcc standard library implementation -std=libstdc++. The following works:

TMB:::compile("VAST_v8_0_0.cpp", CXX="clang++ -stdlib=libstdc++ -std=c++98")

Is this an option on Mac as well?

kaskr avatar Feb 14 '20 15:02 kaskr

First of all, I would like to express my appreciation for your great and long effort for the development of TMB. It is really useful.

As for your comment that the problem is not limited to Mac. I agree with it. Before I posted my comment, Itested to use clang++ on Ubuntu 18.04 but I could not find an option to force to use libc++ there

As for your suggestion to use "-stdlib=libstdc++ -std=c++98” for Mac.

As far as I checked, unfortunately "-stdlib=libstdc++" is not available for clang++ on Mac.

By the way as FAQ of CppAD https://coin-or.github.io/CppAD/doc/faq.htm pointed

"On the other hand, the C++ standard only specifies std::complex<Type> where Type is float, double, or lone double. The effect of instantiating the template complex for any other type is unspecified. “

Probably developers of llvm as for C++ standard library, in particular “complex” header are doing proper job to comply C++ standard.

It seems that use of complex<AD> on MacOS requires to force to use g++ compiler with libstd++ (not a wrapper of clang++ as provided by Apple, but a real g++ from homebrew or somewhere else) at least for now.

2020/02/15 0:13、kaskr [email protected]のメール:

@yukio-takeuchi https://github.com/yukio-takeuchi The problem is not limited to Mac. I can replicate the issue on a Linux system using any version of clang up to 11.0.0. (However, the LLVM version of the C++ standard library -stdlib has to be explicitely selected):

TMB:::compile("VAST_v8_0_0.cpp", CXX="clang++ -stdlib=libc++ -std=c++98") To 'fix' the compilation I switch to the gcc standard library implementation -std=libstdc++. The following works:

TMB:::compile("VAST_v8_0_0.cpp", CXX="clang++ -stdlib=libstdc++ -std=c++98") Is this an option on Mac as well?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kaskr/adcomp/issues/297?email_source=notifications&email_token=AAZ3FVXVBBMDA5QG4PBQ2MTRC2YKFA5CNFSM4HXC5URKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELZK7LY#issuecomment-586330031, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ3FVWX6UNBDK6RF7MH6WDRC2YKFANCNFSM4HXC5URA.

yukio-takeuchi avatar Feb 15 '20 04:02 yukio-takeuchi