cesium-unreal icon indicating copy to clipboard operation
cesium-unreal copied to clipboard

Use (non-Material) Blueprints to pass metadata to Materials

Open kring opened this issue 3 years ago • 2 comments

After #698, Cesium for Unreal has the ability to pass feature metadata into Materials as textures 🎉. While the "Auto Fill" button on the CesiumEncodedMetadataComponent only discovers numeric properties, even string properties can be coerced to numbers (i.e.: parsed) and then used for shading.

However, it's not currently possible to do more complicated transformations of properties to prepare them for use in Materials. For example, Cesium OSM Buildings has a cesium#color property with string values like "rgb(100, 65, 170)". There's no way to parse this string and write an actual color into the generated texture. I hacked up a way to do this in https://github.com/CesiumGS/cesium-unreal/tree/metadata-colors, but I don't think we should ship it in this form.

In CesiumJS, 3D Tiles Styling allows the use of JavaScript-like expressions to style. Implementing this in Unreal at runtime is unlikely, but we could offer an in-editor option to turn a style description into a Material.

Perhaps the more Unreal-y way to do it, though, is to let the user write a Blueprint that takes feature metadata as input and returns values to be stored in a texture, which can then be accessed from a Material. Conceptually this is really straightforward, and a proof of concept of this would probably be quick to implement. But it raises some questions:

  • Where do the Blueprints live? On the encoded metadata component?
  • Will executing a Blueprint function per feature be too slow?
  • Can we run a Blueprint function in a worker thread, so we don't have to block the main thread while we're processing metadata for shading?

kring avatar Apr 25 '22 04:04 kring

Related to #771.

kring avatar May 09 '22 12:05 kring

We considered potentially incorporating this into the metadata overhaul, but we probably won't implement it due to the time constraints and performance concerns. But to quote @kring,

Even just the ability to write and register encoding functions in C++, and then select which one of them to use in the CesiumEncodedMetadataComponent UI, would be huge.

Inspired by this idea, I'd like to rework the encoded metadata component UI as so:

  • Display the property's actual type, regardless of whether or not it is supported.
  • Have some option that displays a method of coercion? Like an enum dropdown that says "None, Uint8, Float." We can even throw in "Color" based on the metadata-color branch, handling rgb values as well as hexcode colors.

This way, it more accurately reflects the data that is available, even if it's incompatible with the GPU. And if the user chooses a coercion method that isn't compatible with that type, maybe we can make use of templates to indicate that it can't be coerced?

That's what I aim to implement for now. This probably won't close the issue, should this implementation make it in, but it's hopefully a good start.

j9liu avatar Aug 01 '23 13:08 j9liu