float icon indicating copy to clipboard operation
float copied to clipboard
verified publisher icon wrathematics

gfortran warnings on macOS

Open PeteHaitch opened this issue 8 years ago • 15 comments

Following from https://github.com/wrathematics/spm/issues/2#issuecomment-319073693, here's the full output. I'm using Apple's Accelerate BLAS (without fully understanding why, except 'it'll be faster')

> devtools::install_github("wrathematics/spm", force = TRUE)
Using GitHub PAT from envvar GITHUB_PAT
Downloading GitHub repo wrathematics/spm@master
from URL https://api.github.com/repos/wrathematics/spm/zipball/master
Installing spm
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file  \
  --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/private/var/folders/f1/6pjy5xbn0_9_7xwq6l7fj2yc0000gn/T/RtmpKUmZrY/devtools452cfdf648/wrathematics-spm-a504dec'  \
  --library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library'  \
  --install-tests 

* installing *source* package ‘spm’ ...
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
BLAS_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRblas
LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack
checking for echo... yes
checking for grep... yes
checking for g77... no
checking for xlf... no
checking for f77... no
checking for frt... no
checking for pgf77... no
checking for cf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for xlf90... no
checking for f90... no
checking for pgf90... no
checking for pghpf... no
checking for epcf90... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran 77 compiler... yes
checking whether gfortran accepts -g... yes
Using internal single precision blas
Using internal single precision lapack
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gfortran   -fPIC  -g -O2  -c lapack/slapack1.f -o lapack/slapack1.o
lapack/slapack1.f:41212:2:

 #if defined(_OPENMP)
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41214:2:

 #endif
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41414:2:

 #if defined(_OPENMP)
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41422:2:

 #endif
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41463:2:

 #if defined(_OPENMP)
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41486:2:

 #else
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41492:2:

 #endif
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41503:2:

 #if defined(_OPENMP)
  1
Warning: Illegal preprocessor directive
lapack/slapack1.f:41506:2:

 #endif
  1
Warning: Illegal preprocessor directive
gfortran   -fPIC  -g -O2  -c lapack/slapack2.f -o lapack/slapack2.o
gfortran   -fPIC  -g -O2  -c lapack/slapack3.f -o lapack/slapack3.o
gfortran   -fPIC  -g -O2  -c lapack/slapack4.f -o lapack/slapack4.o
gfortran   -fPIC  -g -O2  -c lapack/slamchf77.f -o lapack/slamchf77.o
gfortran   -fPIC  -g -O2  -c lapack/ilas.f -o lapack/ilas.o
gfortran   -fPIC  -g -O2  -c lapack/sblas.f -o lapack/sblas.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c bracket.c -o bracket.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c chol.c -o chol.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c converters.c -o converters.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c crossprod.c -o crossprod.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c diag.c -o diag.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c dims.c -o dims.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c isSymmetric.c -o isSymmetric.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c matmult.c -o matmult.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c numbytes.c -o numbytes.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c solve.c -o solve.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c xpose.c -o xpose.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang4/lib -o spm.so lapack/slapack1.o lapack/slapack2.o lapack/slapack3.o lapack/slapack4.o lapack/slamchf77.o lapack/ilas.o lapack/sblas.o bracket.o chol.o converters.o crossprod.o diag.o dims.o isSymmetric.o matmult.o numbytes.o solve.o xpose.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: warning: could not create compact unwind for _spbtrf_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _shseqr_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _sgbtrf_: stack subq instruction is too different from dwarf stack size
installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/library/spm/libs
** R
** tests
** byte-compile and prepare package for lazy loading
Creating a generic function for ‘crossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘tcrossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘diag’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘object.size’ from package ‘utils’ in package ‘spm’
** help
No man pages found in package  ‘spm’ 
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (spm)
Reloading installed spm

> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] spm_0.1-0          lubridate_1.6.0    bindrcpp_0.2      
[4] dplyr_0.7.2        readr_1.1.1        googlesheets_0.2.2
[7] colorout_1.1-2     repete_0.0.0.9008  devtools_1.13.2   

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12            git2r_0.19.0            compiler_3.4.0         
 [4] cellranger_1.1.0        pryr_0.1.2              plyr_1.8.4             
 [7] GenomeInfoDb_1.13.4     XVector_0.17.0          bindr_0.1              
[10] bitops_1.0-6            tools_3.4.0             zlibbioc_1.23.0        
[13] digest_0.6.12           evaluate_0.10.1         debugme_1.0.2          
[16] memoise_1.1.0           tibble_1.3.3            pkgconfig_2.0.1        
[19] rlang_0.1.1.9000        reprex_0.1.1            curl_2.8.1             
[22] yaml_2.1.14             parallel_3.4.0          GenomeInfoDbData_0.99.1
[25] knitr_1.16              withr_2.0.0             stringr_1.2.0          
[28] httr_1.2.1.9000         hms_0.3                 S4Vectors_0.15.5       
[31] IRanges_2.11.12         rprojroot_1.2           stats4_3.4.0           
[34] glue_1.1.1              R6_2.2.2                processx_2.0.0.1       
[37] rmarkdown_1.6           callr_1.0.0.9000        whisker_0.3-2          
[40] clipr_0.3.3             purrr_0.2.2.2           magrittr_1.5           
[43] backports_1.1.0         htmltools_0.3.6         scales_0.4.1           
[46] codetools_0.2-15        fortunes_1.5-4          BiocGenerics_0.23.0    
[49] GenomicRanges_1.29.12   assertthat_0.2.0        colorspace_1.3-2       
[52] stringi_1.1.5           RCurl_1.95-4.8          munsell_0.4.3          
[55] crayon_1.3.2

PeteHaitch avatar Jul 31 '17 14:07 PeteHaitch

Thanks!

By chance, did you replace the files in

/Library/Frameworks/R.framework/Resources/lib

that shipped with R from CRAN with Apple's BLAS/LAPACK manually? I ask because that seems reasonable, but I otherwise can't explain the behavior.

wrathematics avatar Jul 31 '17 14:07 wrathematics

Yeah, I did it manually. I think I basically followed these instructions https://cran.r-project.org/bin/macosx/RMacOSX-FAQ.html#Which-BLAS-is-used-and-how-can-it-be-changed_003f or these http://statistics.berkeley.edu/computing/blas

PeteHaitch avatar Jul 31 '17 14:07 PeteHaitch

I think it should be working now. Thanks again!

wrathematics avatar Jul 31 '17 21:07 wrathematics

Confirm that's tidied up almost all of them. A few of ld warnings remain (installing 23ebc19f75b0af1801b21275f8296dcf41146296)

> install_github('wrathematics/spm')
Using GitHub PAT from envvar GITHUB_PAT
Downloading GitHub repo wrathematics/spm@master
from URL https://api.github.com/repos/wrathematics/spm/zipball/master
Installing spm
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file  \
  --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/private/var/folders/f1/6pjy5xbn0_9_7xwq6l7fj2yc0000gn/T/RtmpcdQsLU/devtools2dd46b75800/wrathematics-spm-23ebc19'  \
  --library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library'  \
  --install-tests 

* installing *source* package ‘spm’ ...
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
BLAS_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRblas
LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack
checking for echo... yes
checking for grep... yes
checking for sgemm_ in -lRblas... no
checking for sgetrf_ in -lRlapack... no
checking for g77... no
checking for xlf... no
checking for f77... no
checking for frt... no
checking for pgf77... no
checking for cf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for xlf90... no
checking for f90... no
checking for pgf90... no
checking for pghpf... no
checking for epcf90... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran 77 compiler... yes
checking whether gfortran accepts -g... yes
Using internal single precision blas
Using internal single precision lapack
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack1.f -o lapack/slapack1.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack2.f -o lapack/slapack2.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack3.f -o lapack/slapack3.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack4.f -o lapack/slapack4.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slamchf77.f -o lapack/slamchf77.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/ilas.f -o lapack/ilas.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/sblas.f -o lapack/sblas.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c bracket.c -o bracket.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c chol.c -o chol.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c converters.c -o converters.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c crossprod.c -o crossprod.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c diag.c -o diag.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c dims.c -o dims.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c isSymmetric.c -o isSymmetric.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c matmult.c -o matmult.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c numbytes.c -o numbytes.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c rand.c -o rand.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c solve.c -o solve.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c xpose.c -o xpose.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang4/lib -o spm.so lapack/slapack1.o lapack/slapack2.o lapack/slapack3.o lapack/slapack4.o lapack/slamchf77.o lapack/ilas.o lapack/sblas.o bracket.o chol.o converters.o crossprod.o diag.o dims.o isSymmetric.o matmult.o numbytes.o rand.o solve.o xpose.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -fopenmp -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: warning: could not create compact unwind for _spbtrf_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _shseqr_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _sgbtrf_: stack subq instruction is too different from dwarf stack size
installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/library/spm/libs
** R
** tests
** byte-compile and prepare package for lazy loading
Creating a generic function for ‘crossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘tcrossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘diag’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘object.size’ from package ‘utils’ in package ‘spm’
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (spm)

PeteHaitch avatar Aug 01 '17 03:08 PeteHaitch

So it's not finding the single precision symbols in your libraries for some reason. These are the relevant lines:

checking for sgemm_ in -lRblas... no
checking for sgetrf_ in -lRlapack... no

This doesn't make sense to me. The BLAS/LAPACK libs it's detecting are:

BLAS_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRblas
LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack

Now I know for a fact that you have the necessary functions, because you were able to build the package before I added all of those fortran files as a backup (you shouldn't be building them, given your setup). I've tested something similar on a local Linux setup and things worked fine, so I'm not sure what the deal is. But I'd bet that if you run:

nm -D /Library/Frameworks/R.framework/Resources/lib/libRblas.so | grep sgemm

that you will get a match.

It's possible that Macs don't respect LDFLAGS, like that weird LD_LIBRARY_PATH vs DYLD_LIBRARY_PATH business. I'll try to look into it, but I'm kind of stumped.

wrathematics avatar Aug 01 '17 12:08 wrathematics

Turns out /Library/Frameworks/R.framework/Resources/lib/libRblas.so doesn't exist! Unsure if this is due to my manual BLAS switch

Peter$ ls -lsh /Library/Frameworks/R.framework/Resources/lib/
total 22536
6760 -rwxrwxr-x  1 root  admin   3.3M 21 Apr 16:42 libR.dylib
   0 drwxrwxr-x  3 root  admin   102B 13 Apr 22:15 libR.dylib.dSYM
 376 -rwxrwxr-x  1 root  admin   185K 21 Apr 16:42 libRblas.0.dylib
   8 lrwxr-xr-x  1 root  admin    16B 25 Apr 15:09 libRblas.dylib -> libRblas.0.dylib
   0 drwxrwxr-x  3 root  admin   102B 13 Apr 22:15 libRblas.dylib.dSYM
4256 -rwxrwxr-x  1 root  admin   2.1M 21 Apr 16:42 libRlapack.dylib
   0 drwxrwxr-x  3 root  admin   102B 13 Apr 22:15 libRlapack.dylib.dSYM
2360 -rwxrwxr-x  1 root  admin   1.1M 21 Apr 16:42 libc++.1.dylib
 704 -rwxrwxr-x  1 root  admin   350K 21 Apr 16:42 libc++abi.1.dylib
 544 -rwxrwxr-x  1 root  admin   271K 21 Apr 16:42 libgcc_s.1.dylib
3160 -rwxrwxr-x  1 root  admin   1.5M 21 Apr 16:42 libgfortran.3.dylib
1728 -rwxrwxr-x  1 root  admin   864K 21 Apr 16:42 libomp.dylib
 560 -rwxrwxr-x  1 root  admin   278K 21 Apr 16:42 libquadmath.0.dylib
1952 -rwxrwxr-x  1 root  admin   973K 21 Apr 16:42 libreadline.5.2.dylib
   8 lrwxr-xr-x  1 root  admin    21B 25 Apr 15:09 libreadline.dylib -> libreadline.5.2.dylib
 120 -rwxrwxr-x  1 root  admin    60K 21 Apr 16:42 libunwind.1.dylib

PeteHaitch avatar Aug 01 '17 12:08 PeteHaitch

Ah no, that's my bad. I forgot macs use .dylib. The configure script is agnostic to extensions, fortunately.

So the thing to try would be:

nm -D /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib | grep sgemm

If that doesn't have a match, then I'm going to be quite confused how you ever installed it in the first place 😅

wrathematics avatar Aug 01 '17 13:08 wrathematics

Ugh, turns out this is probably because I misremembered how I set up this machine.

I repeated the steps in http://statistics.berkeley.edu/computing/blas and now get:

Peter$ ls -lsh /Library/Frameworks/R.framework/Resources/lib/
total 22544
6760 -rwxrwxr-x  1 root   admin   3.3M 21 Apr 16:42 libR.dylib
   0 drwxrwxr-x  3 root   admin   102B 13 Apr 22:15 libR.dylib.dSYM
 376 -rwxrwxr-x  1 root   admin   185K 21 Apr 16:42 libRblas.0.dylib
   8 lrwxr-xr-x  1 Peter  admin   123B  1 Aug 09:37 libRblas.dylib -> /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/libBLAS.dylib
   8 lrwxr-xr-x  1 root   admin    16B 25 Apr 15:09 libRblas.dylib.bak -> libRblas.0.dylib
   0 drwxrwxr-x  3 root   admin   102B 13 Apr 22:15 libRblas.dylib.dSYM
4256 -rwxrwxr-x  1 root   admin   2.1M 21 Apr 16:42 libRlapack.dylib
   0 drwxrwxr-x  3 root   admin   102B 13 Apr 22:15 libRlapack.dylib.dSYM
2360 -rwxrwxr-x  1 root   admin   1.1M 21 Apr 16:42 libc++.1.dylib
 704 -rwxrwxr-x  1 root   admin   350K 21 Apr 16:42 libc++abi.1.dylib
 544 -rwxrwxr-x  1 root   admin   271K 21 Apr 16:42 libgcc_s.1.dylib
3160 -rwxrwxr-x  1 root   admin   1.5M 21 Apr 16:42 libgfortran.3.dylib
1728 -rwxrwxr-x  1 root   admin   864K 21 Apr 16:42 libomp.dylib
 560 -rwxrwxr-x  1 root   admin   278K 21 Apr 16:42 libquadmath.0.dylib
1952 -rwxrwxr-x  1 root   admin   973K 21 Apr 16:42 libreadline.5.2.dylib
   8 lrwxr-xr-x  1 root   admin    21B 25 Apr 15:09 libreadline.dylib -> libreadline.5.2.dylib
 120 -rwxrwxr-x  1 root   admin    60K 21 Apr 16:42 libunwind.1.dylib

So now it's actually using Accelerate's libBLAS whereas it wasn't before (99% sure, anyway). I misremembered the setup of this machine. And when installing spm (full output at end):

checking for sgemm_ in -lRblas... yes
checking for sgetrf_ in -lRlapack... no

So yay, spm is picking up the Accelerate BLAS. But nothing for LAPACK?

Also, I can't find sgemm in either BLAS dylib - in fact, I can't find anything?

Peter$ nm -D /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib | grep sgemm
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib: File format has no dynamic symbol table.
# And not in the original version either
Peter$ nm -D /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib.bak
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib.bak: File format has no dynamic symbol table.

Sorry for the noise on this issue - it's almost certainly caused by the bizarro world set up on my Mac. Out of interest, have you been able to test on a bog-standard macOS set up?

> install_github("wrathematics/spm", force = TRUE)
Using GitHub PAT from envvar GITHUB_PAT
Downloading GitHub repo wrathematics/spm@master
from URL https://api.github.com/repos/wrathematics/spm/zipball/master
Installing spm
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file  \
  --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/private/var/folders/f1/6pjy5xbn0_9_7xwq6l7fj2yc0000gn/T/RtmpWpd85I/devtoolsba725900e9d/wrathematics-spm-23ebc19'  \
  --library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library'  \
  --install-tests 

* installing *source* package ‘spm’ ...
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
BLAS_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRblas
LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack
checking for echo... yes
checking for grep... yes
checking for sgemm_ in -lRblas... yes
checking for sgetrf_ in -lRlapack... no
checking for g77... no
checking for xlf... no
checking for f77... no
checking for frt... no
checking for pgf77... no
checking for cf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for xlf90... no
checking for f90... no
checking for pgf90... no
checking for pghpf... no
checking for epcf90... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran 77 compiler... yes
checking whether gfortran accepts -g... yes
Using system single precision blas
Using internal single precision lapack
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack1.f -o lapack/slapack1.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack2.f -o lapack/slapack2.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack3.f -o lapack/slapack3.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slapack4.f -o lapack/slapack4.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/slamchf77.f -o lapack/slamchf77.o
gfortran  -fopenmp -fPIC  -g -O2  -c lapack/ilas.f -o lapack/ilas.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c bracket.c -o bracket.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c chol.c -o chol.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c converters.c -o converters.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c crossprod.c -o crossprod.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c diag.c -o diag.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c dims.c -o dims.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c isSymmetric.c -o isSymmetric.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c matmult.c -o matmult.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c numbytes.c -o numbytes.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c rand.c -o rand.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c solve.c -o solve.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG   -I/usr/local/include  -fopenmp -fPIC  -Wall -g -O2  -c xpose.c -o xpose.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang4/lib -o spm.so lapack/slapack1.o lapack/slapack2.o lapack/slapack3.o lapack/slapack4.o lapack/slamchf77.o lapack/ilas.o bracket.o chol.o converters.o crossprod.o diag.o dims.o isSymmetric.o matmult.o numbytes.o rand.o solve.o xpose.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -fopenmp -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: warning: could not create compact unwind for _spbtrf_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _shseqr_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _sgbtrf_: stack subq instruction is too different from dwarf stack size
installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/library/spm/libs
** R
** tests
** byte-compile and prepare package for lazy loading
Creating a generic function for ‘crossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘tcrossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘diag’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘object.size’ from package ‘utils’ in package ‘spm’
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (spm)

PeteHaitch avatar Aug 01 '17 13:08 PeteHaitch

The internet seems to be suggesting using nm -gu (or possibly nm -gU) instead of nm -D for macs. Unfortunately I can't test this myself at the moment. One of these days I'm just going to break down and set up a permanent mac server...

Apparently some BLAS vendors put the LAPACK symbols in the BLAS library itself (which is news to me!). OpenBLAS does this, and it's possible that Apple's Accelerate is doing this. This may explain how you were able to build the package before I added the ability to build them from source if they were missing (unless you also changed libRlapack.dylib at some point). So you were getting the single precision LAPACK symbols resolved from linking with your libRblas.dylib. Maybe?

So it might be interesting to test:

# or whatever nm flags actually work
nm -gu /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib | grep gesdd
nm -gu /Library/Frameworks/R.framework/Resources/lib/libRlapack.dylib | grep gesdd

I think you will see s, d, c, z matches for the first one, but only d and z matches for the second. If not, then this really is a mystery :sweat_smile:

This has been extremely helpful to me, so I really appreciate all the help and patience!

wrathematics avatar Aug 02 '17 13:08 wrathematics

So no hits in libRblas.dylib but the expected hits for libRlapack.dylib

# The old (default) BLAS I was using
Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib.bak | grep gesdd
# The Accelerate BLAS that I am using
Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib | grep gesdd
# The unchanged LAPACK
Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRlapack.dylib | grep gesdd
00000000000f7150 T _dgesdd_
000000000018c0e0 T _zgesdd_

I can post the full list of routines if that is helpful, but

Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib | wc -l
     804
$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRlapack.dylib | wc -l
     551

I just noticed I have /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/libLAPACK.dylib. I'm pretty certain I was not using this but it does contain the expected routines

# The Accelerate LAPACK
Peter$ nm -gU /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/libLAPACK.dylib | grep gesdd
00000000000278fb T _cgesdd
00000000000278fb T _cgesdd_
0000000000129a55 T _dgesdd
0000000000129a55 T _dgesdd_
0000000000230657 T _sgesdd
0000000000230657 T _sgesdd_
000000000032af32 T _zgesdd
000000000032af32 T _zgesdd_

I tried to explicitly use it via the same trick I used for switching out the BLAS dylib:

ln -s /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/libLAPACK.dylib libRlapack.dylib

but bad things happened on starting R

Error: package or namespace load failed for ‘stats’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/stats/libs/stats.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/stats/libs/stats.so, 6): Library not loaded: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/stats/libs/stats.so
  Reason: Incompatible library version: stats.so requires version 3.4.0 or later, but libLAPACK.dylib provides version 1.0.0
During startup - Warning message:
package ‘stats’ in options("defaultPackages") was not found

PeteHaitch avatar Aug 02 '17 18:08 PeteHaitch

I have same issue with installation on OS X. Accelerate BLAS is simlinked to libRblas, Lapack is default R's lapack (because linkage of accelerate lapack produce "stats.so requires version 3.4.0 or later, but libLAPACK.dylib provides version 1.0.0")

Here is full log. Not sure why configure decided to use libRlapack:

NOTE: using internal single precision lapack LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack

dselivanov avatar Oct 09 '17 10:10 dselivanov

At the moment, the configure script looks for sgetrf_() (Fortran symbol SGETRF()), which isn't detected in Accelerate. I don't have easy access to a Mac, so I can't test this. Looking into it is definitely on my radar though.

What's stranger is that your compiler and libc are missing a bunch of C99 functions.

wrathematics avatar Oct 10 '17 12:10 wrathematics

I think in that sense configure works correctly - on my system lRlapack is what was shipped with R binaries from CRAN (it is not replaced/simlinked by accelerate lapack).

Does following string mean that package is trying use lapack from package lapack subdir?

NOTE: using internal single precision lapack

For me this looks like it didn't find single precision system lapack, but still tries to link to it (but i'm noob in these configure things)..

LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack

Let me know if I can help somehow.

dselivanov avatar Oct 10 '17 12:10 dselivanov

Yes, the message means it's using the package's internal single precision LAPACK. It's not well worded. I'll try to fix it.

The package ships the float BLAS and LAPACK source since many will only have the double precision symbols. There's obviously a huge advantage in using optimized BLAS (not the ones in the package, which are the "reference" implementations from netlib). For LAPACK it may not matter much from a run time perspective. However, the compilation time will be significantly longer if it has to compile LAPACK, so it's still desirable to find those symbols if they're there.

So currently the configure script looks at the BLAS library linked with R for single precision BLAS symbols, and it looks at the LAPACK library and the BLAS library for single precision LAPACK symbols (since Accelerate and OpenBLAS include LAPACK symbols in the "BLAS" library). This works on my Linux box with OpenBLAS, but Accelerate is still a bit of a mystery to me.

wrathematics avatar Oct 10 '17 12:10 wrathematics

I just reinstalled R with brew: brew install r. It installs it in a way R uses accelerate BLAS and LAPACK. So now package installation goes very smoothly.

Installation also works with R from CRAN, but there are several warnings at the end:

ld: warning: ld: warning: could not create compact unwind for shseqr: stack subq instruction is too different from dwarf stack sizecould not create compact unwind for spbtrf: stack subq instruction is too different from dwarf stack size

ld: warning: could not create compact unwind for sgbtrf: stack subq instruction is too different from dwarf stack size installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/library/float/libs

Don't know what they are about.

dselivanov avatar Nov 08 '17 08:11 dselivanov