Download models in the background

[nkoenig]

Signed-off-by: Nate Koenig [email protected]

🎉 New feature

Closes #1260.

This is an alternative to #1484.

Summary

The PR changes the Server load to:

1. Disable Fuel downloads.
2. Parse the SDF file/string.
3. Remove all models, actors, and lights from each world.
4. Create the worlds (runners).
5. Enable Fuel downloads and force simulation to be paused. User cannot unpause.
6. Start a thread that parses the SDF file again. This time models are downloaded.
7. Add the models back into the worlds.
8. Allow the pause state in simulation to change.

The above will be followed even if nothing needs to be downloaded.

I think the biggest downside to this approach is the double parsing of SDF.

Test it

I haven't added tests yet. Looking for general feedback first.

Checklist

• [ ] Signed all commits for DCO
• [ ] Added tests
• [ ] Added example and/or tutorial
• [ ] Updated documentation (as needed)
• [ ] Updated migration guide (as needed)
• [ ] Consider updating Python bindings (if the library has them)
• [ ] codecheck passed (See contributing)
• [ ] All tests passed (See test coverage)
• [ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

[ahcorde]

When I restart the simulation some models disappear (even if it's not a fuel model)

[nkoenig]

WorldByName is not defined, is there any other related PR ?

Here it is: https://github.com/gazebosim/sdformat/pull/1121

[nkoenig]

When I restart the simulation some models disappear (even if it's not a fuel model)

Does this still happen to you? I've made some changes. If so, what command did you run on the command line?

[ahcorde]

When I restart the simulation some models disappear (even if it's not a fuel model)

    [
      
        ![background_reset](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
      
    ](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
    
    
      
        
          
        
        
          
          
        
      
      [
        
          
        
      ](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
    
   [ ![background_reset](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif) ](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
  
    [
      
        ![background_reset](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
      
    ](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
    
    
      
        
          
        
        
          
          
        
      
      [
        
          
        
      ](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)
    
   [ ](https://user-images.githubusercontent.com/1933907/187023007-57048dbe-2450-4031-ad96-c364ba218578.gif)

Does this still happen to you? I've made some changes. If so, what command did you run on the command line?

@nkoenig I'm getting a crash, I can see the follwoing messages:

Msg [NameManager::issueNewName] (World::Skeleton | shapes) The name [ground_plane] is a duplicate, so it has been renamed to [ground_plane(1)]
Msg [NameManager::issueNewName] (World::Skeleton | shapes) The name [box] is a duplicate, so it has been renamed to [box(1)]
Msg [NameManager::issueNewName] (World::Skeleton | shapes) The name [cylinder] is a duplicate, so it has been renamed to [cylinder(1)]
Msg [NameManager::issueNewName] (World::Skeleton | shapes) The name [sphere] is a duplicate, so it has been renamed to [sphere(1)]
Msg [NameManager::issueNewName] (World::Skeleton | shapes) The name [capsule] is a duplicate, so it has been renamed to [capsule(1)]
Msg [NameManager::issueNewName] (World::Skeleton | shapes) The name [ellipsoid] is a duplicate, so it has been renamed to [ellipsoid(1)]

...

[GUI] [Err] [SceneManager.cc:207] Visual: [ground_plane] already exists
[GUI] [Err] [SceneManager.cc:207] Visual: [box] already exists
[GUI] [Err] [SceneManager.cc:207] Visual: [cylinder] already exists
[GUI] [Err] [SceneManager.cc:207] Visual: [sphere] already exists
[GUI] [Err] [SceneManager.cc:207] Visual: [capsule] already exists
[GUI] [Err] [SceneManager.cc:207] Visual: [ellipsoid] already exists
[GUI] [Err] [BaseStorage.hh:927] Another item already exists with name: sun

and this is the backtrace

Stack trace (most recent call last) in thread 230483:
#26   Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in 
#25   Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7fec81f5c132, in clone
#24   Object "/lib/x86_64-linux-gnu/libpthread.so.0", at 0x7fec81e22608, in 
#23   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7ad069d1, in 
#22   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7ad05784, in QThread::exec()
#21   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7aecd3aa, in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>)
#20   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7af26434, in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
#19   Object "/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0", at 0x7fec78d0d4a2, in g_main_context_iteration
#18   Object "/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0", at 0x7fec78d0d3ff, in 
#17   Object "/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0", at 0x7fec78d0d17c, in g_main_context_dispatch
#16   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7af26e36, in 
#15   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7aed1487, in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*)
#14   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7aece809, in QCoreApplication::notifyInternal2(QObject*, QEvent*)
#13   Object "/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5", at 0x7fec7a66a0ef, in QApplication::notify(QObject*, QEvent*)
#12   Object "/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5", at 0x7fec7a660a65, in QApplicationPrivate::notify_helper(QObject*, QEvent*)
#11   Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7aefac29, in QObject::event(QEvent*)
#10   Object "/home/ahcorde/gz_garden/install/lib/gz-gui-7/plugins/libMinimalScene.so", at 0x7fec45688348, in gz::gui::plugins::RenderThread::RenderNext(gz::gui::plugins::RenderSync*)
#9    Object "/home/ahcorde/gz_garden/install/lib/gz-gui-7/plugins/libMinimalScene.so", at 0x7fec45695a2f, in gz::gui::plugins::RenderThreadRhiOpenGL::RenderNext(gz::gui::plugins::RenderSync*)
#8    Object "/home/ahcorde/gz_garden/install/lib/gz-gui-7/plugins/libMinimalScene.so", at 0x7fec4568b459, in gz::gui::plugins::GzRenderer::Render(gz::gui::plugins::RenderSync*)
#7    Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7aece809, in QCoreApplication::notifyInternal2(QObject*, QEvent*)
#6    Object "/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5", at 0x7fec7a66a0ef, in QApplication::notify(QObject*, QEvent*)
#5    Object "/usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5", at 0x7fec7a660a54, in QApplicationPrivate::notify_helper(QObject*, QEvent*)
#4    Object "/usr/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7fec7aece51a, in QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*, QEvent*)
#3    Object "/home/ahcorde/gz_garden/install/lib/gz-sim-7/plugins/gui/libGzSceneManager.so", at 0x7fec42fc1d6f, in gz::sim::v7::GzSceneManager::eventFilter(QObject*, QEvent*)
#2    Object "/home/ahcorde/gz_garden/install/lib/gz-sim-7/plugins/gui/libGzSceneManager.so", at 0x7fec42fc2116, in gz::sim::v7::GzSceneManagerPrivate::OnRender()
#1    Object "/home/ahcorde/gz_garden/install/lib/libgz-sim7-rendering.so.7", at 0x7fec42d3f7d4, in gz::sim::v7::RenderUtil::Update()
#0    Object "/home/ahcorde/gz_garden/install/lib/libgz-sim7-rendering.so.7", at 0x7fec42e33661, in gz::sim::v7::SceneManager::CreateLight(unsigned long, sdf::v13::Light const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 87.43590% with 49 lines in your changes missing coverage. Please review.

Project coverage is 64.74%. Comparing base (5a76b28) to head (cf5f36a). Report is 19 commits behind head on gz-sim7.

:exclamation: Current head cf5f36a differs from pull request most recent head 656f6d6

Please upload reports for the commit 656f6d6 to get more accurate results.

Files	Patch %	Lines
src/ServerPrivate.cc	76.92%	21 Missing :warning:
src/SimulationRunner.cc	83.46%	21 Missing :warning:
src/LevelManager.cc	94.93%	4 Missing :warning:
src/SdfEntityCreator.cc	96.82%	2 Missing :warning:
src/Conversions.cc	0.00%	1 Missing :warning:

Additional details and impacted files

@@             Coverage Diff             @@
##           gz-sim7    #1669      +/-   ##
===========================================
+ Coverage    64.72%   64.74%   +0.01%     
===========================================
  Files          357      357              
  Lines        29160    29192      +32     
===========================================
+ Hits         18874    18900      +26     
- Misses       10286    10292       +6     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[nkoenig]

GUI is blocked when the models are being downloaded

Shouldn't the GUI block if you're waiting for assets?

[nkoenig]

I unblocked the GUI when --wait-for-assets is true in 594a509

[nkoenig]

@ahcorde , how do you reset simulation?

[nkoenig]

@ahcorde , I added in your resource spawner updates in 3e3a362

[nkoenig]

Reset should now work.