gazebo-classic
gazebo-classic copied to clipboard
gazebosim
Gazebo 11 does not run correctly with Ogre 1.12 from conda-forge
Original report (archived issue) by Silvio Traversaro (Bitbucket: traversaro).
There have been two reports of problems in running Gazebo compiled against Ogre 1.12, one by Sean Yen (seanyen-msft) ( https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/issues/2686/terrain-rendering-does-not-support-shadows (#2686)#comment-56374947 ) and the other by Wolf Vollprecht (Wolf) ( https://github.com/conda-forge/gazebo-feedstock/pull/11#issuecomment-596067386 ), and in both cases the error message is:
[Err] [..\gazebo\gui\main.cc:34] Ogre Error:Ogre::RuntimeAssertionException::RuntimeAssertionException: Ogre/ShadowExtrudePointLight not found. Verify that you referenced the 'ShadowVolume' folder in your resources.cfg in Ogre::ShadowVolumeExtrudeProgram::initialise at ..\OgreMain\src\OgreShadowVolumeExtrudeProgram.cpp (line 71)
In both cases, they were using the Ogre 1.12 installed by conda-forge. Based on this error, and the fact that apparently Stephen Just ( https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/pull-requests/3130/change-gazebo-rendering-heightmap-class-to/diff#comment-125305344, https://github.com/microsoft/vcpkg/pull/8178#issuecomment-552752416 ) was able to run a patched Gazebo 10 linked with Ogre 1.12 using vcpkg, I suspect that the error is that Ogre is not able to find some resource, possibly because some Ogre resource directory is not correctly set, but this is just an hypothesis.
Based on his work on https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/pull-requests/3130/change-gazebo-rendering-heightmap-class-to/diff#comment-125305344, perhaps Martin Pecka (peci1) may be able to provide some input on this.
Original comment by Silvio Traversaro (Bitbucket: traversaro).
- changed title from "Gazebo 11 does not run correctly with Ogre 1.12" to "Gazebo 11 does not run correctly with Ogre 1.12 from conda-forge"
Original comment by Silvio Traversaro (Bitbucket: traversaro).
If you check the environment used by Stephen Just to run Gazebo with Ogre 1.12 linked in https://github.com/microsoft/vcpkg/pull/8178#issuecomment-552752416, you can notice that he added %gz_root_path%Media to GAZEBO_RESOURCE_PATH. It may be a coincidence, but apparently the ShadowVolume folder that is referenced in the error, is indeed installed in <prefix>/Media by Ogre 1.12 (see https://github.com/OGRECave/ogre/blob/v1.12.0/CMakeLists.txt#L501) and it was not installed by Ogre 1.11 .
Perhaps it may be worth a shot to try to add that directory to GAZEBO_RESOURCE_PATH and see if that solves the problem.
Original comment by Silvio Traversaro (Bitbucket: traversaro).
I managed to run Gazebo built against Ogre 1.12 provided by vcpkg (in particular vcpkg version 2020.01 ) by adding the <ogre_install_prefix>/Media to GAZEBO_RESOURCE_PATH in the setup.bat . I guess a similar solution (or workaround) probably should work also with Ogre 1.12 provided by conda-forge .
Original comment by Silvio Traversaro (Bitbucket: traversaro).
Another (unrelated) problem that was present with Ogre 1.12 is the one mentioned (and fixed) in https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/pull-requests/3204/fix-ogre-fontmanager-getbyname-use-in-ogre/diff .
Original comment by Silvio Traversaro (Bitbucket: traversaro).
This change is explicitly mentioned in the Ogre 1.12 Changelog (https://github.com/OGRECave/ogre/blob/master/Docs/1.12-Notes.md#stable-media-files):
ACTION REQUIRED you will have to add the
Media/ShadowVolumeresource location to use the build-in algorithms.
I recently try again with the latest ogre installed from vcpkg and the latest commit in the gazebo11 branch, and I was unable to reproduce this issue, as gzclient starts correctly even without adding <ogre_install_prefix>/Media to GAZEBO_RESOURCE_PATH. I thought there was strange caching going on my machine, but I tried to export the binaries running on my machine on a new clean Windows laptop, and everything was working fine. Perhaps the fix in https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/pull-requests/3205 had also an influence on this for some reason?
@seanyen @wolfv Just to understand, the gazebo package on conda-forge is still using ogre1.10 ? From https://github.com/conda-forge/gazebo-feedstock/pull/14 it would seem that is using the latest ogre, but I am not 100% sure.
@traversaro There was an attempt to build Gazebo 11 with OGRE 1.12. Despite it is built, I ran into some issues at runtime which seems to be associated with:
ACTION REQUIRED you will have to add the Media/ShadowVolume resource location to use the build-in algorithms.
However, it seems to me that in your testing, it is actually working fine with the Vcpkg OGRE port. I think I might need to retry it again.
Finally, I was able to reproduce the problem on an almost vanilla Ubuntu 20.04, using the Gazebo 11 with ogre 1.12 binary that we recently added in conda-forge (see https://github.com/conda-forge/gazebo-feedstock/pull/85):
(ros2) straversaro@iiticublap103:~/Downloads$ gazebo --verbose
Gazebo multi-robot simulator, version 11.7.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org
[Msg] Waiting for master.
Gazebo multi-robot simulator, version 11.7.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org
[Msg] Waiting for master.
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 192.168.1.59
[Msg] Loading world file [/home/straversaro/mambaforge/envs/ros2/share/gazebo-11/worlds/empty.world]
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 192.168.1.59
[Wrn] [Event.cc:61] Warning: Deleting a connection right after creation. Make sure to save the ConnectionPtr from a Connect call
[Wrn] [GuiIface.cc:120] Qt has caught an exception thrown from an event handler. Throwing
exceptions from an event handler is not supported in Qt.
You must not let any exception whatsoever propagate through Qt code.
If that is not possible, in Qt 5 you must at least reimplement
QCoreApplication::notify() and catch all exceptions there.
[Err] [main.cc:37] Ogre Error:RuntimeAssertionException: Ogre/ShadowExtrudePointLight not found. Verify that you referenced the 'ShadowVolume' folder in your resources.cfg in initialise at /home/conda/feedstock_root/build_artifacts/ogre_1624011825937/work/OgreMain/src/OgreShadowVolumeExtrudeProgram.cpp (line 70)
(ros2) straversaro@iiticublap103:~/Downloads$ conda list
# packages in environment at /home/straversaro/mambaforge/envs/ros2:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_gnu conda-forge
alsa-lib 1.2.3 h516909a_0 conda-forge
argcomplete 1.12.3 pyhd8ed1ab_2 conda-forge
assimp 5.0.1 hedfc422_5 conda-forge
atk-1.0 2.36.0 h3371d22_4 conda-forge
attr 2.4.48 h516909a_0 conda-forge
attrs 21.2.0 pyhd8ed1ab_0 conda-forge
boost 1.74.0 py38hc10631b_3 conda-forge
boost-cpp 1.74.0 h312852a_4 conda-forge
bullet 3.17 ha770c72_0 conda-forge
bullet-cpp 3.17 h1abd341_0 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.17.1 h7f98852_1 conda-forge
ca-certificates 2021.5.30 ha878542_0 conda-forge
cairo 1.16.0 h6cf1ce9_1008 conda-forge
catkin_pkg 0.4.23 pyh9f0ad1d_0 conda-forge
certifi 2021.5.30 py38h578d9bd_0 conda-forge
cffi 1.14.6 py38ha65f79e_0 conda-forge
cfitsio 3.470 hb418390_7 conda-forge
cmake 3.21.0 h8897547_0 conda-forge
console_bridge 1.0.1 h4bd325d_0 conda-forge
cppcheck 2.5 py38hbffb2f6_0 conda-forge
cppzmq 4.7.1 hf7cf922_2 conda-forge
cryptography 3.4.7 py38ha5dfef3_0 conda-forge
curl 7.77.0 hea6ffbf_0 conda-forge
cycler 0.10.0 py_2 conda-forge
dartsim 6.10.1 he39ca3a_1 conda-forge
dbus 1.13.6 h48d8840_2 conda-forge
distro 1.5.0 pyh9f0ad1d_0 conda-forge
docutils 0.17.1 py38h578d9bd_0 conda-forge
eigen 3.3.9 h4bd325d_1 conda-forge
empy 3.3.4 pyh9f0ad1d_1 conda-forge
expat 2.4.1 h9c3ff4c_0 conda-forge
fcl 0.6.1 h2cbc392_3 conda-forge
ffmpeg 4.3.1 hca11adc_2 conda-forge
flake8 3.9.2 pyhd8ed1ab_0 conda-forge
flann 1.9.1 h2e58136_1008 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.13.1 hba837de_1005 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
foonathan-memory 0.6.2 he1b5a44_1 conda-forge
freeglut 3.2.1 h9c3ff4c_2 conda-forge
freeimage 3.18.0 h88c329d_7 conda-forge
freetype 2.10.4 h0708190_1 conda-forge
freexl 1.0.6 h7f98852_0 conda-forge
fribidi 1.0.10 h36c2ea0_0 conda-forge
gazebo 11.7.0 he292fef_1 conda-forge
gdbm 1.18 h0a1914f_2 conda-forge
gdk-pixbuf 2.42.6 h04a7f16_0 conda-forge
geos 3.9.1 h9c3ff4c_2 conda-forge
geotiff 1.6.0 h4f31c25_6 conda-forge
gettext 0.19.8.1 h0b5b191_1005 conda-forge
giflib 5.2.1 h36c2ea0_2 conda-forge
glew 2.1.0 h9c3ff4c_2 conda-forge
glib 2.68.3 h9c3ff4c_0 conda-forge
glib-tools 2.68.3 h9c3ff4c_0 conda-forge
gmock 1.10.0 h4bd325d_7 conda-forge
By manually adding $CONDA_PREFIX/share/OGRE/Media/ShadowVolume to GAZEBO_RESOURCE_PATH, the earlier problem is solved but now I get this error:
(ros2) straversaro@iiticublap103:~/mambaforge/envs/ros2/share/gazebo-11$ gazebo --verbose
Gazebo multi-robot simulator, version 11.7.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org
[Msg] Waiting for master.
Gazebo multi-robot simulator, version 11.7.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org
[Msg] Waiting for master.
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 192.168.1.59
[Msg] Loading world file [/home/straversaro/mambaforge/envs/ros2/share/gazebo-11/worlds/empty.world]
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 192.168.1.59
[Wrn] [Event.cc:61] Warning: Deleting a connection right after creation. Make sure to save the ConnectionPtr from a Connect call
[Wrn] [GuiIface.cc:120] Qt has caught an exception thrown from an event handler. Throwing
exceptions from an event handler is not supported in Qt.
You must not let any exception whatsoever propagate through Qt code.
If that is not possible, in Qt 5 you must at least reimplement
QCoreApplication::notify() and catch all exceptions there.
[Err] [main.cc:37] Ogre Error:RuntimeAssertionException: gpu program could not be created in createGpuPrograms at /home/conda/feedstock_root/build_artifacts/ogre_1624011825937/work/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp (line 251)
(ros2) straversaro@iiticublap103:~/mambaforge/envs/ros2/share/gazebo-11$
I found this but it shouldn't matter as we already have 1.12.12 in conda-forge: https://forums.ogre3d.org/viewtopic.php?f=2&t=96240
Also see https://github.com/osrf/gazebo/issues/2986 where the same error occurs.
Also see #2986 where the same error occurs.
Interesting, this may suggest that is a general problem with Ogre 1.12.12, and not a Conda-specific problem. Indeed, in the vcpkg-based build of Gazebo (that work fine, at least in Windows) we are using Ogre 1.12.9 (https://github.com/microsoft/vcpkg/blob/45fc55825db2a5bcaffccff1e6afadc519d164ea/ports/ogre/vcpkg.json), so this could be a change/regression related to Ogre 1.12.12 . Indeed, looking at https://repology.org/project/ogre/versions the only distros with 1.12.12 seem to be some OpenSUSE one and Rosa Linux, so it is possible that no one else found this problem.
However, I inspected a bit the Ogre code (in https://github.com/OGRECave/ogre/blob/v1.12.12/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp#L251) and the error is related to a shader program that was not able to compile. This is compatible with the fact that the same Ogre version is used in RViz2, and there it seems to work fine, so the problem could be related to some Gazebo-specific shader. So to continue the debugging, we should at least collect some info on which shader compilation is failing, and hopefully on the specific compilation error. Probably this will require to build both Ogre and Gazebo from source.
Hi, I tried building with ogre 1.12.9 with gazebo 11.6.0 but still same issue. Not sure if it is expected from the comment above.
Hi, I tried building with ogre 1.12.9 with gazebo 11.6.0 but still same issue. Not sure if it is expected from the comment above.
Interesting, thanks for reporting. I wonder why instead the 1.12.9 via vcpkg on Windows works fine.
Hi, I tried building with ogre 1.12.9 with gazebo 11.6.0 but still same issue. Not sure if it is expected from the comment above.
Interesting, thanks for reporting. I wonder why instead the 1.12.9 via vcpkg on Windows works fine.
Do you think I can use strace to debug this? Never use it before.
If the shader compilation involve any kind of system call, using strace could indeed provide some useful information, but I do not know enough of shader compilation to know if this is the case before checking.
FYI @mboisson depending on the ogre version you have in Easybuild, you may be affected by this problem as well.
@traversaro thanks for the pointer. I'm not sure I follow everything, but which version of OGRE is safe to use ? I gather 1.12.12 is not. Is all of the 1.12 branch to avoid, and I should build with 1.11 ? or should 1.12.x (for x<12) be fine ?
I'm not sure I follow everything, but which version of OGRE is safe to use ?
I don't know a-priori which version of ogre is safe, I can only tell you that:
- apt on Ubuntu 20.04 uses ogre 1.9.0, and it seems not to be affected by this problem
- The default build of Gazebo on conda-forge use ogre 1.10.12, and they are not affected by this problem
- I have a vcpkg-based build of Gazebo on Windows using ogre 1.12.9, and it is not affected by this problem
- On conda-forge we tried to use 1.12.12 on a variant build, but it is affected by this problem
- According to @kevinsmia1939 , flatpack-based build of Gazebo that use both ogre 1.12.9 or 1.12.12 seems to be affected by this problem
However, it is not obvious to tell which version of Ogre may be affected without understanding the source of the issue.
Ok. I'm gonna go back down to 1.10.12, as it seems issues happen starting in 1.11: https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/issues/2686/page/1?slug=terrain-rendering-does-not-support-shadows#comment-56374947
🤔 actually, ignition-rendering3_3.5.0 does not build with ogre 1.10.12 :( (https://github.com/ignitionrobotics/ign-rendering/issues/374)
🤔 actually, ignition-rendering3_3.5.0 does not build with ogre 1.10.12 :( (https://github.com/ignitionrobotics/ign-rendering/issues/374)
Just FYI, ign-rendering is not a dependency of Classic Gazebo.
Hi,
I use Ogre 1.9.0 and Gazebo 11.7.0, Gazebo does not show
Ogre Error:RuntimeAssertionException: gpu program could not be created in createGpuPrograms at /run/build/OGRE/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp error anymore.
Now Gazebo is working and does not freeze now.
I can keep trying with newer Orge to see which version broke.
I can keep trying with newer Orge to see which version broke.
Great, if you could to this it would be quite useful.
I will keep trying more, here is the test so far. Gazebo 11.7.0
-DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
-DCMAKE_INSTALL_PREFIX:PATH=/app
Ogre
-DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
-DOGRE_INSTALL_DOCS:BOOL=OFF
-DOGRE_BUILD_TESTS:BOOL=OFF
-DOGRE_BUILD_COMPONENT_CSHARP:BOOL=OFF
-DOGRE_BUILD_DEPENDENCIES=FALSE
1.9.0 work
1.10.12 work
1.11.0 work
1.11.2 work
1.11.3 work
1.11.4 broke
[Err] [main.cc:37] Ogre Error:InternalErrorException: Not all parameters could be constructed for the sub-render state. in IntegratedPSSM3::resolveParameters at /run/build/Gazebo/gazebo/rendering/CustomPSSMShadowCameraSetup.cc (line 236)
1.11.5 broke
[Err] [main.cc:37] Ogre Error:InternalErrorException: Not all parameters could be constructed for the sub-render state. in IntegratedPSSM3::resolveParameters at /run/build/Gazebo/gazebo/rendering/CustomPSSMShadowCameraSetup.cc (line 236)
1.11.6 broke
[Err] [main.cc:37] Ogre Error:InternalErrorException: Not all parameters could be constructed for the sub-render state. in IntegratedPSSM3::resolveParameters at /run/build/Gazebo/gazebo/rendering/CustomPSSMShadowCameraSetup.cc (line 236)
1.12.0 work
1.12.3 work
1.12.4 work
1.12.5 broke
[Err] [main.cc:37] Ogre Error:RuntimeAssertionException: gpu program could not be created in createGpuPrograms at /run/build/OGRE/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp (line 252)
1.12.9 broke
Ogre Error:RuntimeAssertionException: gpu program could not be created in createGpuPrograms at /run/build/OGRE/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp (line 251)
1.12.12 broke
Ogre Error:RuntimeAssertionException: gpu program could not be created in createGpuPrograms at /run/build/OGRE/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp (line 251)
So I found out that this bug occurred when Ogre change from 1.12.4 to 1.12.5.