Viewport Renderer for MatchMove

[david-cattermole]

Feature

The goal is to add features to the Maya viewport that improve the quality, and the ability to inspect the scene in helpful ways for use of MatchMove tasks.

This renderer will use Maya's viewport 2.0 rendering, but provide a layer (stack)-based system for compositing sets of objects with different shaders/display properties assigned.

Since this is a layer/stack-based system, the order of the Display layers matters and the layers are composited from bottom-to-top, just like in Photoshop or GIMP.

Each Display layer should have some attributes, such as:

• Blend Mode (Z-Depth, Over, Multiply, Add, etc)
• Color
• Opacity/Alpha
• Display / Shader Override (see below)

Examples of planned Display/Shader overrides are:

• Wireframe Mode.
• Shaded Mode.
• Hidden line Mode.
• Object Silhouette / Edges.
• Facing Ratio.
• "Use Background" shader

All objects in the Display Layer will share the same shader/layer properties. Each object in the scene can only be in one Display Layer, and therefore is rendered only once.

By default the Blend mode will be "Z-Depth", which would composite the layers based on the Z-depth that is rendered in each, however this can be changed to "Over", "Multiply" or "Add". Using these non-Z-Depth modes would allow users to force some objects to be visible in front of others, even if the depth of the object is physically farther away. For example, it would be possible to place image planes on the bottom-most layer which forces them to be drawn and composted on-top of, or we could add an object on a front layer and be sure it will always be on-top of a background.

Usage Steps

1. Switch the Viewport to use the "mmRenderer" renderer.
2. Create Display Layers and assign objects.
3. Dynamic attributes are added to the display layers. Users can change the attributes to change how each display layer is rendered.
4. The viewport updates in realtime. Users can playblast whatever they can see. The system is WYSIWYG.
5. If the user wishes to disable "mmRenderer" simply change the "renderer" back to Viewport 2.0.

If a user opens the Maya file without mmSolver installed, all display layers will still be visible and will retain all metadata.

Software Versions

• mmSolver version: v0.4.x
• Maya version: All supported.
• Operating System (OS): All supported.

[david-cattermole]

Current development of this tool has been happening in the vp_renderer. https://github.com/david-cattermole/mayaMatchMoveSolver/tree/vp_renderer

[david-cattermole]

I've been working on this again recently and I have a beta version of MM Renderer that will be released in "v0.4.4.beta1".

The current bugs/limitations are below.

• Issue The objects in the Display Layer "defaultLayer" are ignored.

  • Workaround: You must create a new Display Layer and add the objects into it.
  • This will be fixed in a later release.

• Display Layers may not have MM Renderer attributes added.

  • In this beta release Display Layers will have attributes automatically added when the viewport renderer is toggled.
  • Workaround: Set your viewport renderer to Viewport 2.0, then back to "MM Renderer (beta)".
  • This will be fixed in a later release.

• The Maya HUD is not displayed.

  • This will be fixed in a later release.

• Performance with many objects and/or Display Layers will reduce the speed of the viewport.

  • This will be fixed in a later release.

• On Maya 2022.0, you may have some incorrect visuals

  • Workaround: Upgrade to at least Maya 2022.1+

• "ZDepth" Layer Mode doesn't work.

  • When you have multiple Display Layers in a scene the order of the layers matters. The objects in each layer are rendered and then composited "over" the previous layer - like a Photoshop layer. This is correct when the Layer mode is "Over", but is not currently working as expected with the "ZDepth" layer mode (which should not do that).
  • This will be fixed in a later release.

[david-cattermole]

In the release v0.4.5.beta1 there are a number of changes to MM Renderer.

The targeted capabilities have been significantly reduced to just support for MM Image Plane nodes to be rendered in the "background", behind objects with the Maya native useBackground shader. This feature is critical and allows users to use MM ImagePlane nodes when rendering "hidden line" effects.

As well as MM ImagePlane supporting the useBackground shader this change also allows any custom plug-in (or shape node) to opt-in for rendering in the "background" by adding the string "imageplane" into the draw classification string that is registered with plugin nodes.

The logic for determining which objects are rendered in the "background" is here: https://github.com/david-cattermole/mayaMatchMoveSolver/blob/master/src/mmSolver/render/ops/scene_utils.cpp#L75

---

While developing the other planned features it became clear that the performance overhead is simply not worth the effort and the overall viewport override was becoming increasingly complex and brittle to develop. As a result I have made the plan to postpone the other features such as:

• Display Layers used as 2D "Photoshop-style" layering effects, including:
  • Layer transparency/alpha
  • Layer display shading mode override.
• 3D or 2D "Edge" silhouette rendering.

Such features may be added in the future.

[david-cattermole]

A bug in v0.5.0.beta3 is showing that Silhouette rendering is not updating each frame when rendering a playblast, but it updates correctly in the viewport.

From testing in Maya 2024, the current frame used when running the playblast is has correct silhouette rendering, but all others are not updated. Likewise, the Maya background colour is incorrectly removed after the first frame of the playblast and is replaced with opaque black.