Qualified names for metadata access in custom shaders

[javagl]

Right now, metadata is passed to the custom shaders via an fsInput.metadata structure that contains the properties. The shader code may access this data, with something like

vec2 example = fsInput.metadata.exampleVector;

where exampleVector is the name of a property in "the" metadata class.

When there are multiple classes, and two of them have a property with the same name, then this cannot be represented with this structure. (Right now, it will causes the shader compilation to bail out with some ~"duplicate field in struct" error message).

---

It will be necessary to disambiguate these names. And this disambiguation also has to take into account the schema (because there might be two schemas with the same class name, where the classes use the same property name).

A straightforward solution could be to refer to the variables with fully qualified names, as

vec2 example = fsInput.metadata.exampleSchemaId.exampleClassName.examplePropertyName;

(where exampleSchemaId is the schema.id). This is complicated on the implementation side, because it requires the creation of some deeply nested structs. Alternatively, the qualification could happen with _ underscore characters, as in

vec2 example = fsInput.metadata.exampleSchemaId_exampleClassName_examplePropertyName;

In both cases, accessing these properties becomes a bit clumsy. But they both could be reasonable ways of having the data in the shader, consistently and unambiguously.

---

<brainstorming>

A shader that uses data similar to the example from the specification README currently looks as follows:

var houseTextureShader = new Cesium.CustomShader({
  lightingModel: Cesium.LightingModel.UNLIT,
  fragmentShaderText: [
    "void fragmentMain(FragmentInput fsInput, inout czm_modelMaterial material)",
    "{",
    "    int inside = fsInput.metadata.insideTemperature;",
    "    int outside = fsInput.metadata.outsideTemperature;",
    "    float insulation = fsInput.metadata.insulation;",
    "    material.diffuse = vec3(float(inside)/255.0, float(outside)/255.0, insulation);",
    "}",
  ].join("\n"),
});

Adding the full qualification would yield

var houseTextureShader = new Cesium.CustomShader({
  lightingModel: Cesium.LightingModel.UNLIT,
  fragmentShaderText: [
    "void fragmentMain(FragmentInput fsInput, inout czm_modelMaterial material)",
    "{",
    "    int inside = fsInput.metadata.exampleSchema.buildingComponents.insideTemperature;",
    "    int outside = fsInput.metadata.exampleSchema.buildingComponents.outsideTemperature;",
    "    float insulation = fsInput.metadata.exampleSchema.buildingComponents.insulation;",
    "    material.diffuse = vec3(float(inside)/255.0, float(outside)/255.0, insulation);",
    "}",
  ].join("\n"),
});

---

Not being deeply involved in the details of the custom shader mechanisms, I wondered whether it could make sense to offer some configurability on this on the API level. Roughly like

houseTextureShader.defineInput("insideTemperature", "exampleSchema.buildingComponents.insideTemperature");
houseTextureShader.defineInput("outsideTemperature", "exampleSchema.buildingComponents.outsideTemperature");
houseTextureShader.defineInput("insulation", "exampleSchema.buildingComponents.insulation");

so that the respective properties can then be accessed with fsInput.outsideTemperature. Possible (idealistic) advantages of something like this could be that

• it would be possible to (verbatim) use the same shader code, regardless of how things are named in the metadata (establishing the "wiring" between the metadata and shader inputs could happen on the API level)
• it might even be possible to implement that in a way that allows switching these inputs at runtime, without having to re-compile the shader (but there might be reasons why this is technically not viable)

</brainstorming>

[ptrgags]

Had a (very rough) idea overnight after seeing some of the details in #10520 and #10569. Styling makes use of a variableSubstitutionMap that takes property names from the styles and translates them to some shader variable.

I wonder if it's possible for custom shaders to just use a flat list of properties in the Metadata struct (instead of nested structs) with unique names (generated automatically, e.g. property_0, property_1) and then use a dictionary or function to map property names (whether fully qualified or not) from the shader to these auto-generated names.

That said, such a scheme would be harder to debug, since the GLSL variable names would not have meaning depending on how vari

I also am wondering how easy it would be to make the variable substitution map, given that properties can come from various places (property tables, property textures, property attributes, tileset/tile/group/content metadata).

Furthermore, it would also be good to think about how this relates to the CPU picking/styling equivalent through Cesium3DTileFeature.getPropertyInherited(). that currently uses a precedence chain to resolve name conflicts (fine-grained metadata is checked before coarse-grained metadata). But perhaps the substitution map could also have a way to look up values (including handling fully-qualified names). See also #10015.

In other words, some sort of MetadataAccessor abstraction where you give

To sketch out some rough pseudocode:

const accessor = model.metadataAccessor;

// For picking/CPU styling (possibly public API?)
accessor.getValue('propertyId') // tries to find the property value but throws(?) if there's a conflict
accessor.getValue('className.propertyId') // a little more specific
accessor.getValue('schemaId.className.propertyId') // very specific

// for custom shaders (this would be private API only)
accessor.getGlslName('propertyId') // returns some auto-generated name, e.g. property_17 or maybe property_propertyId_17 for readability?
// then loop over the custom shader code and regex-replace `metadata.className.propertyId` and so on with `metadata.property_17`

Still a bit cumbersome to use, but it might dodge the nested struct generation at least.

Regardless, this would require more discussion to see what makes sense