Feature styling with metadata

[j9liu]

Cesium for Unity should offer feature styling, similar to Cesium for Unreal. The implications of this affect what we'll do with point cloud rendering, so I'm starting to think about this now. CC @argallegos, we discussed this offline and I'm paraphrasing his ideas for this issue.

In Cesium for Unreal, feature styling with metadata is done through the CesiumEncodedMetadata component. We can see how it's used via the Visualizing Metadata tutorial, but basically it creates nodes in the Unreal version of Shader Graph and uses HLSL to access the metadata properties.

It makes sense to do something similar with Shader Graph in Unity, especially because they offer custom function nodes. The user may need to configure the CesiumMetadata component on the tileset, and then create their own version of the material / shader, in order to hook up the metadata properties to their own code. This may have complications with point cloud shading, but I'll write that up in another issue.

Another option (though less preferable) is to have a metadata styling component that shows the user what metadata properties are present. Then, the user can set the colors (and perhaps other styling) of the metadata properties in the component itself, and Cesium for Unity could translate that into a shader under the hood. This is more approachable for a beginner user who is unfamiliar with writing shaders and just wants to do simple styling (e.g. make all buildings of a type blue), but it doesn't offer freedom for people who want to write more advanced shaders.

Just some ideas for now -- we should try to figure out the ideal UX before thinking about implementation details.

[j9liu]

Starting to think about this issue -- I know that Cesium for Unreal auto-generates a material with nodes that retrieve the metadata for the user. To my knowledge, there is no easy way to create a Shader Graph from script in Unity. I found the source code for the package and looked at the Editor code, to see if I could replicate parts of their code, but all of the necessary classes are inaccessible from outside the package.

I'm not experienced with Reflection, but maybe there's a way to reach these shader graph classes with it. In the worst case scenario, we'll need to write up documentation for users to create their own materials and properly access the metadata. I wonder if the user needs a way to view all of the possible metadata properties in a dataset, from a glance.

[j9liu]

I found out that it's possible to write custom shader GUIs for both regular materials and shader graphs (source). Which gives me an idea for a potential workflow, given that autogenerating a shader graph may not be feasible / maintainable.

1. Add a CesiumEncodedFeaturesMetadata component, similar to the component in Unreal. This will be responsible for actually auto-filling and encoding the property values.
2. Create a custom shader GUI. The user has to manually set this in a shader graph they create, but we can document it in our tutorial.
3. In the GUI, help the user generate custom shader code based on the desired properties. This could involve having fields where you set the name + encoded type of the property. Then, there can be a text field where the user can copy + paste the code into a custom function. (This could also be done on the CesiumEncodedFeaturesMetadata component itself)
4. The custom shader GUI can validate if all the necessary parameters are included for the material. For example, we'd use a naming scheme like PTABLE_tableName_propertyName to store a texture. I think it's possible to search for a parameter name in a material, so we could show in the UI whether or not the user has correctly added all required parameters.

Although the user still has to manually add boilerplate code, this GUI can be informative and help aid the process.

[j9liu]

Came up on the forum: https://community.cesium.com/t/select-a-building-and-change-its-color-in-unity/29479

[j9liu]

Came up again on the forum but regarding point clouds: https://community.cesium.com/t/pointcloud-color-classification-in-unity/29764

[kring]

It just occurred to me while fiddling with Unity shader graphs for a different reason... While Unity doesn't provide any API for programmatically creating shader graphs (as far as we can tell), it does store them as JSON in a text file. So in theory, all we need to do is reverse engineer the JSON format well enough to generate a skeleton shader like we do in Unreal. Put the asset file in the right place and Unity will immediately import it. Unity's own asset backward compatibility should keep this from being too much of a moving target. The format isn't trivial, but the logic of it is immediately apparent so it's probably not too hard to deal with. Overall this wouldn't be very different from using Unreal's (undocumented) API, really.