gazebo-classic
gazebo-classic copied to clipboard
gazebosim
UNIT_Dem_TEST failure on 20.04
Description
The UNIT_Dem_TEST fails for me with a segfault, with the following output:
Start 19: UNIT_Dem_TEST
19/424 Test #19: UNIT_Dem_TEST .........................................***Exception: SegFault 0.46 sec
[==========] Running 10 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 10 tests from DemTest
[ RUN ] DemTest.MisingFile
[Wrn] [SystemPaths.cc:459] File or path does not exist ["/file/shouldn/never/exist.png"] [/file/shouldn/never/exist.png]
[Err] [Dem.cc:78] Unable to open DEM file[/file/shouldn/never/exist.png], check your GAZEBO_RESOURCE_PATH settings.
[ OK ] DemTest.MisingFile (2 ms)
[ RUN ] DemTest.NotDem
ERROR 4: `/home/rich/fedora/gazebo/gazebo-11.1.0/media/materials/scripts/CMakeLists.txt' not recognized as a supported file format.
[Err] [Dem.cc:88] Unable to open DEM file[/home/rich/fedora/gazebo/gazebo-11.1.0/media/materials/scripts/CMakeLists.txt]. Format not recognised as a supported dataset.
[ OK ] DemTest.NotDem (0 ms)
[ RUN ] DemTest.UnsupportedDem
[Err] [Dem.cc:96] Unsupported number of bands in file [/home/rich/fedora/gazebo/gazebo-11.1.0/media/materials/textures/wood.jpg]. Found 3 but only 1 is a valid value.
[ OK ] DemTest.UnsupportedDem (1 ms)
[ RUN ] DemTest.NonSquaredDemPortrait
[ OK ] DemTest.NonSquaredDemPortrait (24 ms)
[ RUN ] DemTest.NonSquaredDemLandscape
[ OK ] DemTest.NonSquaredDemLandscape (12 ms)
[ RUN ] DemTest.SquaredDem
[ OK ] DemTest.SquaredDem (11 ms)
[ RUN ] DemTest.BasicAPI
[Err] [Dem.cc:181] EXCEPTION: Illegal coordinates. You are asking for the elevation in (0,129) but the terrain is [129 x 129]
[Err] [Dem.cc:181] EXCEPTION: Illegal coordinates. You are asking for the elevation in (129,0) but the terrain is [129 x 129]
[Err] [Dem.cc:181] EXCEPTION: Illegal coordinates. You are asking for the elevation in (129,129) but the terrain is [129 x 129]
[ OK ] DemTest.BasicAPI (15 ms)
[ RUN ] DemTest.FillHeightmap
[ OK ] DemTest.FillHeightmap (13 ms)
[ RUN ] DemTest.NegDem
ERROR 1: PROJ: proj_create_operations: Source and target ellipsoid do not belong to the same celestial body
ERROR 6: Cannot find coordinate operations from `PROJCRS["Moon2000_npole",BASEGEOGCRS["GCS_Moon",DATUM["Moon_2000",ELLIPSOID["Moon_2000_IAU_IAG",1737400,0,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Reference_Meridian",0,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]],CONVERSION["unnamed",METHOD["Polar Stereographic (variant B)",ID["EPSG",9829]],PARAMETER["Latitude of standard parallel",90,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8832]],PARAMETER["Longitude of origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8833]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["easting",south,ORDER[1],LENGTHUNIT["metre",1]],AXIS["northing",south,ORDER[2],LENGTHUNIT["metre",1]]]' to `EPSG:4326'
Looking in to the issue, it appears that the call to OGRCreateCoordinateTransformation is returning NULL and printing the above error, but the return result is never checked before using the cT variable.
Additionally, the cT object appears to be leaked as it is never expliclty released using OCTDestroyCoordinateTransformation() per the documentation
Environment
- Fedora 32 x86_64
- gazebo-11.1.0
- proj-6.3.2
I've added a check for a nullptr and fixed the memory leak in a branch in https://github.com/scpeters/gazebo/commit/e9d95a26c0fe5fdeb7d09840f77fa3ec5d8c9df9; can you take a look at that? I think there must be something else wrong though, since I don't see any console output from DemTest.NegDem on Ubuntu 18.04.
That commit looks good to me.
It looks like 18.04 has proj-4.9.3 - it could be a change between that release and Fedora's 6.3.2. Focal has 6.3.1 - maybe there's a way to test it there?
There's some mailing list traffic where people have had similar problems. The error output here definitely looks like it's trying to convert from the moon to the earth.
FWIW I now get this output with the commit you linked (on gazebo 10.1.0). It looks like the test after the segfault is also failing, now that the transform failure throws an exception.
Start 19: UNIT_Dem_TEST
19/408 Test #19: UNIT_Dem_TEST .........................................***Failed 0.44 sec
[==========] Running 10 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 10 tests from DemTest
[ RUN ] DemTest.MisingFile
[Wrn] [SystemPaths.cc:459] File or path does not exist ["/file/shouldn/never/exist.png"] [/file/shouldn/never/exist.png]
[Err] [Dem.cc:78] Unable to open DEM file[/file/shouldn/never/exist.png], check your GAZEBO_RESOURCE_PATH settings.
[ OK ] DemTest.MisingFile (2 ms)
[ RUN ] DemTest.NotDem
ERROR 4: `/home/rich/fedora/gazebo/gazebo-10.1.0/media/materials/scripts/CMakeLists.txt' not recognized as a supported file format.
[Err] [Dem.cc:88] Unable to open DEM file[/home/rich/fedora/gazebo/gazebo-10.1.0/media/materials/scripts/CMakeLists.txt]. Format not recognised as a supported dataset.
[ OK ] DemTest.NotDem (1 ms)
[ RUN ] DemTest.UnsupportedDem
[Err] [Dem.cc:96] Unsupported number of bands in file [/home/rich/fedora/gazebo/gazebo-10.1.0/media/materials/textures/wood.jpg]. Found 3 but only 1 is a valid value.
[ OK ] DemTest.UnsupportedDem (0 ms)
[ RUN ] DemTest.NonSquaredDemPortrait
[ OK ] DemTest.NonSquaredDemPortrait (64 ms)
[ RUN ] DemTest.NonSquaredDemLandscape
[ OK ] DemTest.NonSquaredDemLandscape (47 ms)
[ RUN ] DemTest.SquaredDem
[ OK ] DemTest.SquaredDem (47 ms)
[ RUN ] DemTest.BasicAPI
[Err] [Dem.cc:181] EXCEPTION: Illegal coordinates. You are asking for the elevation in (0,129) but the terrain is [129 x 129]
[Err] [Dem.cc:181] EXCEPTION: Illegal coordinates. You are asking for the elevation in (129,0) but the terrain is [129 x 129]
[Err] [Dem.cc:181] EXCEPTION: Illegal coordinates. You are asking for the elevation in (129,129) but the terrain is [129 x 129]
[ OK ] DemTest.BasicAPI (60 ms)
[ RUN ] DemTest.FillHeightmap
[ OK ] DemTest.FillHeightmap (47 ms)
[ RUN ] DemTest.NegDem
ERROR 1: PROJ: proj_create_operations: Source and target ellipsoid do not belong to the same celestial body
ERROR 6: Cannot find coordinate operations from `PROJCRS["Moon2000_npole",BASEGEOGCRS["GCS_Moon",DATUM["Moon_2000",ELLIPSOID["Moon_2000_IAU_IAG",1737400,0,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Reference_Meridian",0,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]],CONVERSION["unnamed",METHOD["Polar Stereographic (variant B)",ID["EPSG",9829]],PARAMETER["Latitude of standard parallel",90,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8832]],PARAMETER["Longitude of origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8833]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["easting",south,ORDER[1],LENGTHUNIT["metre",1]],AXIS["northing",south,ORDER[2],LENGTHUNIT["metre",1]]]' to `EPSG:4326'
[Err] [Dem.cc:220] EXCEPTION: Unable to transform terrain coordinate system to WGS84 for coordinates (0,0)
unknown file: Failure
Unknown C++ exception thrown in the test body.
[ FAILED ] DemTest.NegDem (13 ms)
[ RUN ] DemTest.UnfinishedDem
/home/rich/fedora/gazebo/gazebo-10.1.0/gazebo/common/Dem_TEST.cc:219: Failure
Value of: demNoData.GetWorldHeight()
Actual: 7499.8281
Expected: 14050.08
/home/rich/fedora/gazebo/gazebo-10.1.0/gazebo/common/Dem_TEST.cc:220: Failure
Value of: demNoData.GetWorldWidth()
Actual: 14150.225
Expected: 9785.4375
[ FAILED ] DemTest.UnfinishedDem (41 ms)
[----------] 10 tests from DemTest (322 ms total)
[----------] Global test environment tear-down
[==========] 10 tests from 1 test case ran. (322 ms total)
[ PASSED ] 8 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] DemTest.NegDem
[ FAILED ] DemTest.UnfinishedDem
2 FAILED TESTS
I'm not sure if it's related, but there's also a compiler warning in Dem.cc:
gazebo/common/Dem.cc: In member function ‘void gazebo::common::Dem::GetGeoReference(double, double, ignition::math::v6::Angle&, ignition::math::v6::Angle&) const’:
/root/gazebo/gazebo/common/Dem.cc:215:41: warning: ‘OGRErr OGRSpatialReference::importFromWkt(char**)’ is deprecated [-Wdeprecated-declarations]
215 | sourceCs.importFromWkt(&importString);
| ^
In file included from /root/gazebo/gazebo/common/Dem.cc:25:
/usr/include/gdal/ogr_spatialref.h:204:17: note: declared here
204 | OGRErr importFromWkt( char ** )
| ^~~~~~~~~~~~~
the compiler warning should be fixed by https://github.com/osrf/gazebo/commit/ad932ed368eb4aa0c93ca5538f2415dcca959e8f
I think the following line may be the issue:
- https://github.com/scpeters/gazebo/blob/e9d95a26c0fe5fdeb7d09840f77fa3ec5d8c9df9/gazebo/common/Dem.cc#L216
I've found the place in proj that throws the "Source and target ellipsoid do not belong to the same celestial body" exception:
- https://github.com/OSGeo/PROJ/blob/6.3.2/src/iso19111/coordinateoperation.cpp#L13509-L13514
support for ISO 19111 was added in proj 6.0.0:
- https://github.com/OSGeo/PROJ/blob/6.0.0/NEWS#L10
I added some code that prints warning messages instead of failing hard on 20.04 when this type of failure occurs. The UNIT_Dem_TEST still fails on 20.04, but it doesn't throw an exception
Hello ! I'm testing on Ubuntu 20.04 with proj 6.3.1 , and the test does not fail with that particular error. I'll try testing inside a docker container.
I meant to add a comment after updating the title of this issue. The test was recently updated in #3257, and it passes on our primary Ubuntu CI on 18.04 but fails on 20.04:
- https://build.osrfoundation.org/job/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/48/testReport/junit/(root)/DemTest/LunarDemLoad/
19: [ RUN ] DemTest.NegDem
19: ERROR 1: PROJ: proj_create_operations: Source and target ellipsoid do not belong to the same celestial body
19: ERROR 6: Cannot find coordinate operations from `PROJCRS["Moon2000_npole",BASEGEOGCRS["GCS_Moon",DATUM["Moon_2000",ELLIPSOID["Moon_2000_IAU_IAG",1737400,0,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Reference_Meridian",0,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]],CONVERSION["unnamed",METHOD["Polar Stereographic (variant B)",ID["EPSG",9829]],PARAMETER["Latitude of standard parallel",90,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8832]],PARAMETER["Longitude of origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8833]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["easting",south,ORDER[1],LENGTHUNIT["metre",1]],AXIS["northing",south,ORDER[2],LENGTHUNIT["metre",1]]]' to `EPSG:4326'
19: [1;31m[Err] [Dem.cc:265] [0m[1;31mEXCEPTION: [0m[1;31mUnable to transform terrain coordinate system to WGS84 for coordinates (0,0)
19: [0m[1;31m
19: [0m[1;31m[0m[1;33m[Wrn] [Dem.cc:141] [0m[1;33mFailed to automatically compute DEM size. [0m[1;33mPlease use the <size> element to manually set DEM size.[0m[1;33m
19: [0m[1;33m[0m/home/jenkins/workspace/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/gazebo-classic/gazebo/common/Dem_TEST.cc:183: Failure
19: Expected equality of these values:
19: 293.51068
19: dem.GetWorldHeight()
19: Which is: 0
19: /home/jenkins/workspace/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/gazebo-classic/gazebo/common/Dem_TEST.cc:184: Failure
19: Expected equality of these values:
19: 293.51089
19: dem.GetWorldWidth()
19: Which is: 0
19: [ FAILED ] DemTest.NegDem (35 ms)
19: [ RUN ] DemTest.UnfinishedDem
19: /home/jenkins/workspace/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/gazebo-classic/gazebo/common/Dem_TEST.cc:220: Failure
19: Expected equality of these values:
19: 14050.08
19: demNoData.GetWorldHeight()
19: Which is: 7499.8281
19: /home/jenkins/workspace/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/gazebo-classic/gazebo/common/Dem_TEST.cc:221: Failure
19: Expected equality of these values:
19: 9785.4375
19: demNoData.GetWorldWidth()
19: Which is: 14150.225
19: [ FAILED ] DemTest.UnfinishedDem (73 ms)
19: [ RUN ] DemTest.LunarDemLoad
19: ERROR 1: PROJ: proj_create_operations: Source and target ellipsoid do not belong to the same celestial body
19: ERROR 6: Cannot find coordinate operations from `PROJCRS["Moon2000_npole",BASEGEOGCRS["GCS_Moon",DATUM["Moon_2000",ELLIPSOID["Moon_2000_IAU_IAG",1737400,0,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Reference_Meridian",0,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]],CONVERSION["unnamed",METHOD["Polar Stereographic (variant B)",ID["EPSG",9829]],PARAMETER["Latitude of standard parallel",90,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8832]],PARAMETER["Longitude of origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8833]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["easting",south,ORDER[1],LENGTHUNIT["metre",1]],AXIS["northing",south,ORDER[2],LENGTHUNIT["metre",1]]]' to `EPSG:4326'
19: [1;31m[Err] [Dem.cc:265] [0m[1;31mEXCEPTION: [0m[1;31mUnable to transform terrain coordinate system to WGS84 for coordinates (0,0)
19: [0m[1;31m
19: [0m[1;31m[0m[1;33m[Wrn] [Dem.cc:141] [0m[1;33mFailed to automatically compute DEM size. [0m[1;33mPlease use the <size> element to manually set DEM size.[0m[1;33m
19: [0m[1;33m[0m/home/jenkins/workspace/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/gazebo-classic/gazebo/common/Dem_TEST.cc:239: Failure
19: The difference between 293.51 and dem.GetWorldWidth() is 3.1709585737814474e+180, which exceeds 0.1, where
19: 293.51 evaluates to 293.50999999999999,
19: dem.GetWorldWidth() evaluates to 3.1709585737814474e+180, and
19: 0.1 evaluates to 0.10000000000000001.
19: /home/jenkins/workspace/gazebo-ci-gazebo11-focal-amd64-gpu-nvidia/gazebo-classic/gazebo/common/Dem_TEST.cc:240: Failure
19: The difference between 293.51 and dem.GetWorldHeight() is 5.5560359219961525e+180, which exceeds 0.1, where
19: 293.51 evaluates to 293.50999999999999,
19: dem.GetWorldHeight() evaluates to 5.5560359219961525e+180, and
19: 0.1 evaluates to 0.10000000000000001.
19: [ FAILED ] DemTest.LunarDemLoad (67 ms)
Two of those failures, NegDem and LunarDem are stemming from the fact that coordinates are transformed differently when the surfaces do not match by different version of libproj. The older version defaults appears to default to Earth, and newer ones give either infinities or zero as sizes.
I've added a workaround for those 2 here : https://github.com/gazebosim/gazebo-classic/pull/3275
Okay, so I did a bit of troubleshooting in Ubuntu 18.04(libproj version ??) vs Ubuntu 20.04 (libproj 6.3.1):
- For
LunarDemandNegDem: ** Troublesome line of code : https://github.com/gazebosim/gazebo-classic/blob/d86d435c7b153a0c182ff61a62916970437e59bd/gazebo/common/Dem.cc#L262
Here, In Ubuntu 18 (and older libproj I assume), the cT pointer returns a valid pointer, assuming the surface was Earth, and no warning is thrown. In Ubuntu 20, and newer libproj, cT is a null pointer with the console spamming about mismatched coordinates. The tests fail due to different paths followed based on whether cT is null or not, as that is the only warning we have that the surfaces are mismatched.
** Solution : What I can do right now is modify the tests so that the old libproj defaults to Earth, and newer ones default to 0 world heights and widths (as is intended in the code)
- For
UnfinishedDem: ** Troublesome line : https://github.com/gazebosim/gazebo-classic/blob/d86d435c7b153a0c182ff61a62916970437e59bd/gazebo/common/Dem.cc#L272
Here, in Ubuntu 20.04, the xGeoDeg and yGeoDeg variables have their values interchanged after the statement cT->Transform(...). It seems like a known issue : https://github.com/OSGeo/gdal/issues/1546