Importing glTF Materials into MaterialX

[emackey]

When importing a glTF model, it may be desirable to import the materials as well. This is made easier now with the recently added <gltf_pbr> node from #861. Here I'll gather my notes on how this might be done.

The obvious first step when encountering a glTF material is to create a <gltf_pbr> node connected to a <surfacematerial> output (I'm going to ignore volumetric for this first pass). The glTF material may have "factors" and "textures" included, where a "factor" roughly corresponds to a preset uniform value. Various factors can, in the abscence of a texture, have their factor values simply copied to the <gltf_pbr> node:

From glTF materials block	To <gltf_pbr> node
pbrMetallicRoughness.baseColorFactor[0, 1, 2]	base_color
pbrMetallicRoughness.baseColorFactor[3]	alpha
pbrMetallicRoughness.metallicFactor	metallic
pbrMetallicRoughness.roughnessFactor	roughness
emissiveFactor	emissive
extensions.KHR_materials_transmission.transmissionFactor	transmission
extensions.KHR_materials_specular.specularFactor	specular
extensions.KHR_materials_specular.specularColorFactor	specular_color
extensions.KHR_materials_ior.ior	ior
extensions.KHR_materials_sheen.sheenColorFactor	sheen_color
extensions.KHR_materials_sheen.sheenRoughnessFactor	sheen_roughness
extensions.KHR_materials_clearcoat.clearcoatFactor	clearcoat
extensions.KHR_materials_clearcoat.clearcoatRoughnessFactor	clearcoat_roughness
extensions.KHR_materials_volume.thicknessFactor	thickness
extensions.KHR_materials_volume.attenuationDistance	attenuation_distance
extensions.KHR_materials_volume.attenuationColor	attenuation_color

When a "texture" version of any of the above parameters is present, a <tiledimage> node should be created for it, ideally using the tiling strategy given in the glTF's associated texture sampler block (default is to REPEAT texture coordinates).

If both a "factor" and a "texture" are present, they are multipled together. In the case of color textures (baseColor, emissive, specularColor, sheenColor), the texture should undergo sRGB-to-Linear transform prior to being multiplied by the factor. I think (but I'm not sure) this can be done by adjusting the type attribute on <tiledimage>. Use color3 for images that glTF treats as colors (listed above), and vector3 for the rest.

For the case of pbrMetallicRoughness.metallicRoughnessTexture, the image must be fed into an <extract> node to pull out the green channel for roughness and the blue channel for metallic. There's an example of this in #870 in gltf_pbr_boombox.mtlx.

Several other images have preassigned channels. If pbrMetallicRoughness.baseColorTexture contains an alpha channel, it must be extracted, RGB to base_color and the alpha channel to alpha. occlusion always comes from the red channel. We can look up the rest, but know that all single-channel float inputs in glTF have specified channel mappings for their source textures. Each one should use <extract> to ensure it is reading the appropriate single channel.

The normalTexture and clearcoatNormalTexture images must be fed to <normalmap> node(s) prior to being connected to their respective inputs. The normal map space is always tangent in glTF. The <normalmap> node also has a scale input that can receive the value of glTF's normalTexture.scale or extensions.KHR_materials_clearcoat.clearcoatNormalTexture.scale setting.

Ideally, running this process on the glTF BoomBox sample should produce a node graph quite similar to gltf_pbr_boombox.mtlx in #870.

I'll write a separate issue for export to glTF some other time.

[donmccurdy]

If pbrMetallicRoughness.baseColorTexture contains an alpha channel, it must be extracted...

Probably conditional on alphaMode ≠ "OPAQUE" for this step.

Is it necessary to flag vertex colors, normals, and tangents on the material?

[emackey]

Good condition for alpha, thanks.

I'm not sure how MaterialX handles vertex colors, maybe there's a node for them? If so, the importer could multiply that node by the baseColor / alpha inputs.

[kwokcb]

I've moved my sample code to use cgltf and a draft is here.

There are obviously many things missing from it and the logic is partial, but a start.

Note: For vertex colors there are geomcolor nodes which I mention here for a possible way to map "unlit" shader as this has been added to MaterialX recently.

[emackey]

Awesome start, I will need some time for a closer look in the next day or two.

For vertex colors there are geomcolor nodes which I mention here for a possible way to map "unlit" shader as this has been added to MaterialX recently.

In glTF, there's an extension KHR_materials_unlit. If you find this extension on a particular material, that material should avoid using <gltf_pbr> and use your unlit shader instead. The unlit color in the glTF is sRGB-to-linear( baseColorTexture ) * baseColorFactor * vertex_color. Note vertex colors are linear, not sRGB. Any of these inputs may be 3-channel or 4-channel, and in cases where blendMode is anything but the default OPAQUE, the fourth channel becomes the material's alpha input. Missing channels/inputs are considered 1.0.

If there's no unlit extension on a material, the same baseColor/vertex color calculation is used, but fed into the <gltf_pbr> node's base_color and alpha inputs for normal PBR rendering.

In glTF, when not supplied, baseColor, roughness, metallic, and occlusion all default to 1.0. The default values of <gltf_pbr> may need adjusting to accommodate that, or you can have the importer assign these values explicitly.

[kwokcb]

FYI: To allow for independent repo development, I've migrated the basic glTF->MTLX code to this repo, but can move it wherever folks deem is best. There is basic ASCII glTF conversion working. It uses an externally defined location for build and running the simple command line program. (glTF2Mtlx). It is still easier to work in a MaterialX fork repo vs this independent one but it's just 1 class where all the logic resides.

[kwokcb]

As a discussion point for MTLX->glTF.

• I have tried performing this conversion but I would propose that the configuration created from glTF to MTLX be the only ones supported and any other graph configurations should be distilled to this target graph.
• The experiments continue in this PR , where I am reverse translating the MTLX document created with forward translation as input. Anyways this experiment has been done to open up discussion here :).

[kwokcb]

House cleaning update:

I have migrated my personal branch found in this PR to the Khronos fork here.

@jstone-lucasfilm, I have some fixes for the geometry loader to migrate back which fixes pathing, but breaks assignments to the default shaderball. This we discussed before but either the files using looks needs to change, or the glTF files does. I think the latter makes more sense -- the transform hierarchy just needs to be collapsed.

@ashwinbhat , I noticed Christian Robles references the personal branch. Please reference the new one. Also do you know who created the glTF file for the default shader ball. Was it Sebastian Dunkel ?

The prototype repo I have not updated yet as want to discuss how to proceed with dependencies.

[kwokcb]

@emackey , @jstone-lucasfilm I believe we can close this now.

I have finished both import (this issue) and export (distillation) in my own repo here which uses the Autodesk contributed translation graph by @ashwinbhat et al.

• There is just one remaining PR for baking under review. If / when this is in then this code can run off of MaterialX main (this repo) without patches.
• I will ping refenced folks off-line as well about possible avenues to make this "official" if desired.

[jstone-lucasfilm]

Thanks @kwokcb and @emackey, this has been a valuable discussion, and I'll go ahead and close out the issue.