MaterialX Template Specification Proposal

[ld-kerley]

Overview

This proposal aims to define the specification langauge that would be added to the MaterialX specification to describe the new template syntax that was introduced in this PR (https://github.com/AcademySoftwareFoundation/MaterialX/pull/2362).

Details

The templating PR describes and implements a system intended to simplify the authoring of the MaterialX standard data library, making it less repetitive, and consequently less error-prone. The system introduces new syntax to the XML-based MaterialX grammar to provide the ability to describe families of nodes in a single "templated" definition, reducing the need to explicitly state each different node signature that exists for a given node.

The PR above explicitly intends for this to new grammar to be introduced in the standard data library source files, and concretely NOT in any MaterialX data library files that are built or installed by the CMake based build process. This approach has the advantage of not requiring that any downstream integrations of MaterialX have knowledge of this update to the grammar, as all templates can be expanded at build time. This is a deliberate design choice of the PR, as discussed in the previous proposal and associated TSC meeting discussions, as it will allow the project the freedom to iterate on the grammar without affecting downstream MaterialX integrations. As such the original PR did NOT intend the formal MaterialX specification be updated initially, as the generated data library files should be unchanged by this process. We are initially just introducing an optimization of the sources used to generate the data library during the build.

Once the PR has been merged, the new build-time template grammar has been given time to mature, and feedback gathered from MaterialX stakeholders, we may decide to introduce a runtime based system. If so, then at that point, the new template grammar would need to be part of the formal specification of MaterialX. This would infer that downstream integrators of MaterialX would be required to support ingestion of a data library containing this new formalized template syntax, if they wanted to provide 100% MaterialX compliance.

Here we attempt to describe formally that syntax at is currently stands. We should node that its possible the MaterialX project may opt to change this before it becomes formalized, or perhaps even decide that the template syntax should remain solely the domain of the build-time files, and that downstream consumers prefer to ingest the fully expanded data library files.

Specification Text

The following section is the proposed text that could be added to the MaterialX specification to support a runtime implementation of the template syntax.

Named Values

The MaterialX data library contains many nodes that have inputs with a number of common semantic default values, for instance zero or one. The MaterialX data library also uses a number of different concrete types, for example float, color3 or vector4. These types each have different concrete values to represent these semantic values. <typedef> elements can be extended with attributes defining these named values. The attribute name should be the name of the named value and the value of the attribute should be the concrete value to be used any place the name is referenced.

  <typedef name="float" zero="0.0" one="1.0"/>
  <typedef name="color3" zero="0.0,0.0,0.0" one="1.0,1.0,1.0"/>
  <typedef name="vector4" zero="0.0,0.0,0.0,0.0" one="1.0,1.0,1.0,1.0"/>
  <typedef name="string" zero=""/>

These named values can be used anywhere a concrete value is accepted, ie. in any value or default attribute. The named value is referenced by prefixing the name of the named value with "Value:".

  <input name="in1" type="color3" value="Value:zero"/>
  <input name="in2" type="float" value="Value:one"/>

This example, when combined with the example <typedef> elements above is functionally identical to:

  <input name="in1" type="color3" value="0.0,0.0,0.0"/>
  <input name="in2" type="float" value="1.0"/>

Not every <typedef> is required to define each named value. Expansion of a named value will raise an error if a specified name is not defined for the given <typedef>.

Templated Definition Elements

To reduce potential repetitive data library elements, any number of elements can be enclosed within a <template> element scope. The elements inside the <template> scope are concretely instantiated using the provided template attributes.

Attributes for <template> elements:

• name (string, required): a unique name for this <template>
• varName (string, required): the name of a variable to be used for substitution in the contained elements.
• options (stringarray, required): the set of concrete values to be used during the substitution of the specified varName in elements contained within the <template>.

When a <template> is expanded, all of the child elements are instantiated exactly once for each entry in the options array. These instances replace the original <template> element in the document. They are placed at the same scope level as the original <template> element in the document.

Each instance of the contents contained inside teh <template> are processed as they are instantiated. This processing step inspects all attribute values in all the contained elements and replaces any occurance of a specific string with the corresponding value from the options array. The specific string used for the string replacement matching is constructed by wrapping the value of varName with @ characters. For example if varName has the value "foo", then the matching string would be "@foo@". We surround the value of varName with the @ characters to ensure we do not have any accidental matches.

  <template name="TP_ND_multiply" varName="typeName" options="float, color3, vector4">
    <nodedef name="ND_multiply_@typeName@" node="multiply" nodegroup="math">
      <input name="in1" type="@typeName@" value="Value:zero" />
      <input name="in2" type="@typeName@" value="Value:one" />
      <output name="out" type="@typeName@" defaultinput="in1" />
    </nodedef>
  </template>

In this example the provided varName is "typeName", and so the matching string is "@typeName@". Each instance of this string is then replaced by each element in the options string array, "float", "color3" and "vector4".

  <nodedef name="ND_multiply_float" node="multiply" nodegroup="math">
    <input name="in1" type="float" value="Value:zero" />
    <input name="in2" type="float" value="Value:one" />
    <output name="out" type="float" defaultinput="in1" />
  </nodedef>
  <nodedef name="ND_multiply_color3" node="multiply" nodegroup="math">
    <input name="in1" type="color3" value="Value:zero" />
    <input name="in2" type="color3" value="Value:one" />
    <output name="out" type="color3" defaultinput="in1" />
  </nodedef>
  <nodedef name="ND_multiply_vector4" node="multiply" nodegroup="math">
    <input name="in1" type="vector4" value="Value:zero" />
    <input name="in2" type="vector4" value="Value:one" />
    <output name="out" type="vector4" defaultinput="in1" />
  </nodedef>

When used in combination with the Named Value expansion, this example would become the following concrete set of fully expanded elements.

  <nodedef name="ND_multiply_float" node="multiply" nodegroup="math">
    <input name="in1" type="float" value="0.0" />
    <input name="in2" type="float" value="1.0" />
    <output name="out" type="float" defaultinput="in1" />
  </nodedef>
  <nodedef name="ND_multiply_color3" node="multiply" nodegroup="math">
    <input name="in1" type="color3" value="0.0,0.0,0.0" />
    <input name="in2" type="color3" value="1.0,1.0,1.0" />
    <output name="out" type="color3" defaultinput="in1" />
  </nodedef>
  <nodedef name="ND_multiply_vector4" node="multiply" nodegroup="math">
    <input name="in1" type="vector4" value="0.0,0.0,0.0,0.0" />
    <input name="in2" type="vector4" value="1.0,1.0,1.0,1.0" />
    <output name="out" type="vector4" defaultinput="in1" />
  </nodedef>

[jstone-lucasfilm]

@ld-kerley This looks like a great starting point, and I would say that the ideal platform for reviewing and ultimately merging this work would be a pull request that adds your new text to the appropriate sections of the Specification Proposals document.

[kwokcb]

Some initial comments / questions.

1. I can see the need to have a placeholder to replace values with the per-type values. I am curious is this a pre-cursor to creating a new "constant" Element or purely template only resolver mechanism?

   • For instance if it's long term constant introduction I could replacement attribute listings like this:

    <typedef name="float" zero="0.0" one="1.0"/>
   

   into concrete Elements as scoped children:

      <typedef name="float">
          <const name="zero" value="0.0"/>
          <const name="one" value="1.0"/>
      </typedef>
   

   • or top level or any other GraphElement level

   <const name="pi" type="float" value="3.14159"/>
   <const name="e" type="float" value="2.71828"/>
   

   • This would allow us to add more constants in the future without changing the schema.
   • References could then be to true Elements instead of strings which may be error prone and mutable.
   • You could also scope constants if desired (e.g. within a nodegraph)

   <input name="on1" type="float" constant="pi"/>
   

   or using your value assignment approach

    <input name="on1" type="float" Value="Constant:pi"/>
   

2. For the templating step, is it worth also taking a more Element based / "overload" approach kind of akin to OSL overloads. Overloads are scoped. Of course this is more intrusive as it injects "overloads" within nodedefs.

<template name="TP_ND_multiply" varName="typeName" options="float, color3, vector4">
  <nodedef name="ND_multiply" node="multiply" nodegroup="math">
    <overload types="@options@">
      <input name="in1" type="@typeName@" value="Constant:zero"/>
      <input name="in2" type="@typeName@" value="Constant:one"/>
      <output name="out" type="@typeName@" defaultinput="in1"/>
    </overload>
  </nodedef>
</template>

Just some thoughts.