Load plugins from <visual><plugin>

[scpeters]

Part of https://github.com/ignitionrobotics/ign-gazebo/issues/265

Desired behavior

From https://github.com/ignitionrobotics/ign-gazebo/issues/265:

Some more pointers for visual plugins that are purely related to rendering:

• ign-gazebo should load <visual><plugin>s. As a reference, here's what it took to load <sensor><plugin>s.
• Gazebo-classic's visual plugins are loaded by gazebo::rendering on both server and client scenes. I believe an equivalent implementation would have to live in ign-rendering.

Alternatives considered

Implementation suggestion

Additional context

[chapulina]

Some extra notes:

• ign-rendering doesn't depend on SDFormat. We can either add this dependency, or use plain XML to pass configuration like ign-gui does

• Some interesting plugin points would be to provide pre-render and render callbacks

• For completeness, I'll just mention that we should provide complete plugin tooling like we do for other libraries, i.e. expose an environment variable with lookup paths for the plugins, etc.

[scpeters]

@iche033 and I just discussed this briefly during a meeting. I think he would prefer that this happen in ign-gazebo instead, which is fine with me. The following comment from @chapulina in https://github.com/ignitionrobotics/ign-gazebo/issues/265#issuecomment-665331412 led me to open this issue:

• Gazebo-classic's visual plugins are loaded by gazebo::rendering on both server and client scenes. I believe an equivalent implementation would have to live in ign-rendering.

The important thing to me is that visual plugins take effect on both the server and GUI client. I can close this issue in favor of https://github.com/ignitionrobotics/ign-gazebo/issues/265 if this can be accomplished there

[chapulina]

this happen in ign-gazebo instead

I'm also ok with that. We already support accessing the rendering API from both server and GUI plugins: https://ignitionrobotics.org/api/gazebo/7.0/rendering_plugins.html. It would be nice to support plugins that are attached to a visual and can be loaded on both the server and client, without the need to create separate ones.

[iche033]

some thoughts on where to implement visual plugins

ign-rendering:

• Need to support loading custom plugins
• needs to add tinyxml / sdformat dependency for reading plugin params
• no access to ECM, no concept of models, links, visuals (almost everything is just a visual or sensor in ign-rendering)
• Once plugin support is implemented, should be easier to get them loaded on both server and gui side in ign-gazebo.

ign-gazebo:

• need to figure out a way to load plugins on both server and gui.
• support reading sdformat params
• access to ECM

Currently the main rendering thread are created by either ign-gui's MinimalScene or ign-gazebo's sensors system. Internally ign-rendering does not run a thread or emit render events. One option to do it in ign-rendering would be to emit render events whenever the scene pre / post render occurs, and have visual plugins connect to these events if they want to perform some logic every iteration. While on ign-gazebo, the event system and rendering thread are all set up but we don't have a way to load plugins on both server and gui side.

[chapulina]

access to ECM

The way this is handled on the GUI front is that plain ignition::gui::Plugins don't have access to the ECM, and ignition::gazebo::GuiSystems extend ignition::gui::Plugins providing access to the ECM on top of plan GUI features.

Need to support loading custom plugins

Worth mentioning that ign-rendering already supports one plugin type: rendering engines.

[scpeters]

ign-rendering:

• Need to support loading custom plugins

this will be a bit of work, but I think ign-plugin handles most of the difficulty

• needs to add tinyxml / sdformat dependency for reading plugin params

I believe ign-gui uses tinyxml2 directly to read plugin parameters and doesn't depend on libsdformat, so we could follow that same pattern. Or we could make libsdformat a dependency of a component of ign-rendering but not the core library, like in ign-physics

• no access to ECM, no concept of models, links, visuals (almost everything is just a visual or sensor in ign-rendering)

perhaps we can look at some of the existing visual plugins in gazebo11 and irg_open to see if there are any that depend on Model / Link APIs

• Once plugin support is implemented, should be easier to get them loaded on both server and gui side in ign-gazebo.

👍

ign-gazebo:

• need to figure out a way to load plugins on both server and gui.
• support reading sdformat params
• access to ECM

Currently the main rendering thread are created by either ign-gui's MinimalScene or ign-gazebo's sensors system. Internally ign-rendering does not run a thread or emit render events. One option to do it in ign-rendering would be to emit render events whenever the scene pre / post render occurs, and have visual plugins connect to these events if they want to perform some logic every iteration. While on ign-gazebo, the event system and rendering thread are all set up but we don't have a way to load plugins on both server and gui side.

it sounds like we don't have any ideas about how to load visual plugins on both server and GUI via ign-gazebo? I consider that a core requirement

[chapulina]

we don't have any ideas about how to load visual plugins on both server and GUI via ign-gazebo

I think we won't be able to use the existing plugin base classes on ign-gazebo, System and GuiSystem, because they're tied to either the server or the GUI.

We could create a new ignition::gazebo::rendering::Plugin base class that's loaded (and updated?) by RenderUtil, because RenderUtil is used by both server and GUI to pass information between the ECM and ign-rendering.

[chapulina]

One option to do it in ign-rendering would be to emit render events whenever the scene pre / post render occurs, and have visual plugins connect to these events if they want to perform some logic every iteration.

I've been giving this issue some thought, and I think there's a lot of value in taking the ign-rendering plugin route instead. The events idea from @iche033 sounds reasonable to me, and supporting generic ign-rendering plugins means they can be leveraged by applications other than Gazebo.