terra
terra copied to clipboard
rspatial
Build failure due to mismatching types: `gdal_multidimensional.cpp:581:66: error: cannot convert '__gnu_cxx::__alloc_traits<std::allocator<int>, int>::value_type*' {aka 'int*'} to 'const GInt64*' {aka 'const long long int*'}`
/opt/local/bin/g++-mp-14 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DHAVE_TBB -DTBB -I/opt/local/libexec/onetbb/include -I/opt/local/lib/proj9/include -DHAVE_PROJ_H -I/opt/local/include -I/opt/local/include -I'/opt/local/Library/Frameworks/R.framework/Versions/4.5/Resources/library/Rcpp/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c gdal_multidimensional.cpp -o gdal_multidimensional.o
gdal_multidimensional.cpp: In member function 'bool SpatRaster::readChunkMulti(std::vector<double>&, size_t, size_t, size_t, size_t, size_t)':
gdal_multidimensional.cpp:581:66: error: cannot convert '__gnu_cxx::__alloc_traits<std::allocator<int>, int>::value_type*' {aka 'int*'} to 'const GInt64*' {aka 'const long long int*'}
581 | source[src].m_array->Read(&offset[0], &count[0], &stride[0], NULL, dt, &data[insize], NULL, 0);
In file included from spatVector.h:24,
from spatRaster.h:20,
from gdal_multidimensional.cpp:1:
/opt/local/include/gdal_priv.h:3710:24: note: initializing argument 3 of 'virtual bool GDALMDArray::Read(const GUInt64*, const size_t*, const GInt64*, const GPtrDiff_t*, const GDALExtendedDataType&, void*, const void*, size_t) const'
3710 | const GInt64 *arrayStep, // step in elements
| ~~~~~~~~~~~~~~^~~~~~~~~
gdal_multidimensional.cpp: In member function 'bool SpatRaster::readRowColMulti(size_t, std::vector<std::vector<double> >&, size_t, std::vector<long long int>&, const std::vector<long long int>&)':
gdal_multidimensional.cpp:666:74: error: cannot convert '__gnu_cxx::__alloc_traits<std::allocator<int>, int>::value_type*' {aka 'int*'} to 'const GInt64*' {aka 'const long long int*'}
666 | source[src].m_array->Read(&offset[0], &count[0], &stride[0], NULL, dt, &v[0], NULL, 0);
/opt/local/include/gdal_priv.h:3710:24: note: initializing argument 3 of 'virtual bool GDALMDArray::Read(const GUInt64*, const size_t*, const GInt64*, const GPtrDiff_t*, const GDALExtendedDataType&, void*, const void*, size_t) const'
3710 | const GInt64 *arrayStep, // step in elements
| ~~~~~~~~~~~~~~^~~~~~~~~
make: *** [gdal_multidimensional.o] Error 1
It look like your are on macOS. Which version? This compiles in macOS on CRAN. And which version of GDAL do you have (that is probably the cause)?
GDAL 3.10.3, config is here: https://github.com/macports/macports-ports/blob/d679d85eeb41937115f91e41b39b4b737b99537c/gis/gdal/Portfile
Earlier versions of terra from CRAN built with no issues, this is the first one with such a failure.
This is on 32-bit ppc, so no surprise that nobody saw the error, it is obscured on 64-bit, since incidentally size of int == size of long long int.
OK thanks, that must be it. Do you know if there is a preprocessor check that I can use to detect this particular 32-bit system? It seems that there is no simple "is_64bit" macro across OSs and compilers.
FreeBSD 14.2 i386 gdal-3.11.0
<snip>
c++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I/usr/local/include -DHAVE_PROJ_H -I/usr/local/include -I/usr/local/include -I'/usr/local/lib/R/library/Rcpp/include' -DLIBICONV_PLUG -I/usr/local/include -isystem /usr/local/include -fpic -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DLIBICONV_PLUG -isystem /usr/local/include -c gdal_multidimensional.cpp -o gdal_multidimensional.o
gdal_multidimensional.cpp:581:52: error: cannot initialize a parameter of type 'const GInt64 *' (aka 'const long long *') with an rvalue of type 'value_type *' (aka 'int *')
581 | source[src].m_array->Read(&offset[0], &count[0], &stride[0], NULL, dt, &data[insize], NULL, 0);
| ^~~~~~~~~~
/usr/local/include/gdal_priv.h:3810:24: note: passing argument to parameter 'arrayStep' here
3810 | const GInt64 *arrayStep, // step in elements
| ^
gdal_multidimensional.cpp:666:53: error: cannot initialize a parameter of type 'const GInt64 *' (aka 'const long long *') with an rvalue of type 'value_type *' (aka 'int *')
666 | source[src].m_array->Read(&offset[0], &count[0], &stride[0], NULL, dt, &v[0], NULL, 0);
| ^~~~~~~~~~
/usr/local/include/gdal_priv.h:3810:24: note: passing argument to parameter 'arrayStep' here
3810 | const GInt64 *arrayStep, // step in elements
| ^
2 errors generated.
gmake: *** [/usr/local/lib/R/etc/Makeconf:209: gdal_multidimensional.o] Error 1
ERROR: compilation failed for package 'terra'
Full log: https://pkg-status.freebsd.org/beefy21/data/142i386-default/691901e18f0f/logs/R-cran-terra-1.8.54.log
Does this patch (https://github.com/rspatial/terra/commit/891bb07da1152988225d1f1cfac75aabfa40df62) fix the problem on your systems?
@rhijmans it fixes build on FreeBSD i386 and tests are OK too. I will use this patch to unbreak port on i386 right away.
Thanks
Maybe unrelated: Just spotted a warning that I don't know if it is related to i386 or if it appears on other arches too:
c++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I/usr/local/include -DHAVE_PROJ_H -I/usr/local/include -I/usr/local/include -I'/usr/local/lib/R/library/Rcpp/include' -DLIBICONV_PLUG -I/usr/local/include
-isystem /usr/local/include -fpic -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DLIBICONV_PLUG -isystem /usr/local/include -c gdalio.cpp -o gdalio.o
gdalio.cpp:932:23: warning: implicit conversion from 'unsigned long long' to 'double' changes value from 18446744073709550514 to 18446744073709549568 [-Wimplicit-const-int-float-conversion]
932 | naflag = UINT64_MAX-1101;
| ~ ~~~~~~~~~~^~~~~
1 warning generated.
Does this patch (https://github.com/rspatial/terra/commit/891bb07da1152988225d1f1cfac75aabfa40df62) fix the problem on your systems?
@rhijmans It probably will, but as a side effect it will assume that AIX, Linux or BSD ppc64 systems are 32-bit (__ppc64__ is correct for Darwin, but it is Darwin-specific), as well as riscv64, mips64, loongarch, sparc64, itanium.
Maybe check for pointer size instead of relying on a set of macros?
@nunotexbsd Notice, if you apply that unconditionally, it will have an impact on every 64-bit arch not covered by those macros.
Going with macros, it arguably makes a better sense to detect 32-bit archs instead, since we are not gonna see new 32-bit archs, perhaps, while a new 64-bit arch or a new platform using existing one can come any day. That won’t be robust, but it probably be safer.
I have now changed it to the below. Do you see any downsides?
#include <cstdint>
#if INTPTR_MAX != INT32_MAX
#define IS_64_BIT
#endif
@rhijmans I will try that now.
On a side note, is it actually possible to fix multidimensional for 32-bit platforms (given that the issue is not specific to old macOS) or that’s a no-go?
I have now changed it to the below. Do you see any downsides?
#include <cstdint> #if INTPTR_MAX != INT32_MAX #define IS_64_BIT #endif
The build from the current master 43126ab0fabc2757564bb05994cc683ce70710e5 works for me on ppc.
I have now changed it to the below. Do you see any downsides?
#include <cstdint> #if INTPTR_MAX != INT32_MAX #define IS_64_BIT #endif
With this patch, build back to fail on i386:
c++ -std=gnu++17 -I"/usr/local/lib/R/include" -DNDEBUG -I/usr/local/include -DHAVE_PROJ_H -I/usr/local/include -I/usr/local/include -I'/usr/local/lib/R/library/Rcpp/include' -DLIBICONV_PLUG -I/usr/local/include
-isystem /usr/local/include -fpic -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DLIBICONV_PLUG -isystem /usr/local/include -c gdal_multidimensional.cpp -o gdal_multidimensional.o
gdal_multidimensional.cpp:586:52: error: cannot initialize a parameter of type 'const GInt64 *' (aka 'const long long *') with an rvalue of type 'value_type *' (aka 'int *')
586 | source[src].m_array->Read(&offset[0], &count[0], &stride[0], NULL, dt, &data[insize], NULL, 0);
| ^~~~~~~~~~
/usr/local/include/gdal_priv.h:3810:24: note: passing argument to parameter 'arrayStep' here
3810 | const GInt64 *arrayStep, // step in elements
| ^
gdal_multidimensional.cpp:671:53: error: cannot initialize a parameter of type 'const GInt64 *' (aka 'const long long *') with an rvalue of type 'value_type *' (aka 'int *')
671 | source[src].m_array->Read(&offset[0], &count[0], &stride[0], NULL, dt, &v[0], NULL, 0);
| ^~~~~~~~~~
/usr/local/include/gdal_priv.h:3810:24: note: passing argument to parameter 'arrayStep' here
3810 | const GInt64 *arrayStep, // step in elements
| ^
2 errors generated.
gmake: *** [/usr/local/lib/R/etc/Makeconf:209: gdal_multidimensional.o] Error 1
ERROR: compilation failed for package 'terra'
@barracuda156: it may be possible to fix the code. Not easy for me to do as I do not have a 32-bit machine to test, and docker won't do for this. Suggestions?
I have removed the macro and changed the code to see if that fixes it. I now use and use a different type for "stride": ptrdiff_t instead of GPtrDiff_t. Perhaps ptrdiff_t has the right size as it is defined in a system library, and the typedef of GPtrDiff_t uses a macro that may be incorrect.
Never mind. The change above works on Windows 64 but not on ubuntu and macOS.
@rhijmans Is there something to prevent from using unambiguous types which will just be identical on any platform?
Now using a "long long int" and that compiles on macOS, ubuntu and windows. Perhaps that also works on your systems?
@barracuda156 these definitions are in GDAL, that I do not develop. They are considering changing this. We can go to them for help, but I wanted to try some things first.
Probably if an explicit 64-bit type will be used instead of int there, problem should not arise. Or if size of type should be conditional on bitness, then avoid explicitly-sized types. Upstream should know better what is appropriate, but the current code is just incorrect, otherwise we would not get failures.
Can you give the current version of terra a try?
Ah, sorry, I completely forgot about this. Will do today.
@rhijmans There is no compilation error – which was initially reported – when building from 7810acbf7bab84711629ec41183d18285e0a9517 (so this issue can be considered resolved).
The build fails to link for me now, unfortunately, but it is not terra’s bug – and there is nothing to do about it from your end. The failure is similar to the one here: https://github.com/USDAForestService/gdalraster/issues/713
A quick summary for the reference:
– C++ has two ABIs, independent from C++ standard.
– On older macOS, for compatibility reasons, MacPorts normally uses the old ABI, which is done via the base passing the flag to build systems. However, this is not perfectly consistent, since automatic passing of the flag does not always work, so various ports will have that done manually.
– For historical reasons, R itself and its ecosystem do not use old ABI, regardless of macOS version. This worked well, since R packages seldom link to anything besides R itself, R packages, OS and compiler libraries. In a few cases where ABI conflict arose, I have patched in -D_GLIBCXX_USE_CXX11_ABI=0 manually into Makvars (for example R-magick and R-rjags ports). AFAIK, until R 4.4.x there was no reverse problem: everything either worked as-is using the new ABI (in 99% of cases) or failed on linking to an external library, and then passing -D_GLIBCXX_USE_CXX11_ABI=0 to an R package in question fixed linking.
– Now there is a different issue with some Rcpp-using packages, when -D_GLIBCXX_USE_CXX11_ABI=0 produces broken libs that fail to load. It looks like something in Rcpp and/or related C++-using packages uses ABI-specific stuff which makes them incompatible with the old ABI. I have explained the problem here: https://trac.macports.org/ticket/72494
– I do not see an easy solution, sadly. Of course, in principle everything should consistently use just one ABI, which presumably should remove these issues altogether. In practice, however, neither switching all R ecosystem to the old ABI nor switching everything else to the new ABI is easy or even safe. The only alternative which I can currently think of – which would not require months of re-compilations and testing – is to build isolated duplicates of GEOS, GDAL and a few other C++ ports, which R packages link to, using the new ABI, and then tweak Makevars so that those ones are used and not the regular from the main prefix. It is an ugly hack, but it will probably solve the problem. Until that is done, just peg failing packages to w/e the last version which still builds without ABI conflicts.
I have fixed the issue with ABI conflict: https://github.com/macos-powerpc/R-ports/commit/f3ed2940b7ad1806bdef5c0e638d3e0e3e111b34 https://github.com/macos-powerpc/R-ports/commit/4b561b2021950b46d78c3be1c6da258ae1f9b99e
The original issue is fixed now? If yes, this can be closed.