Research + prototype rendering quality lighting improvements (PBR)

[markschlosseratbentley]

A variety of WebGL rendering improvements could be added beyond the regular lighting implemented by iTwin.js. This could result in a more appealing rendering.

This project is to research + evaluate various aspects of physically based rendering (PBR). We can prototype some PBR techniques using existing rendering packages, a simple standalone sample, or a modified version of iTwin.js. See the discussion on this work item for more details.

Consider:

• ~~Path tracing (Approximation of various advanced lighting techniques):~~
  • ~~https://en.wikipedia.org/wiki/Path_tracing~~
  • ~~https://madebyevan.com/webgl-path-tracing/~~
  • ~~Downside: Could be grainy while navigating view until image settles. Could experiment with various settings to avoid this. (Regular lighting until view is done moving?)~~
  • ~~Consider performance.~~
• Screen space reflections (Approximates surfaces bouncing light in a single pass) (reflectivity):
  • https://threejs.org/examples/webgl_postprocessing_ssr.html
  • (This sample uses water largely, but various surfaces could have a reflectivity property, and all could have a baseline value) https://lettier.github.io/3d-game-shaders-for-beginners/screen-space-reflection.html
• Search for other PBR candidate techniques to improve rendering.

[pmconne]

PBR materials is the standard.

[markschlosseratbentley]

PBR materials is the standard.

@pmconne - @eringram was going to take a look at researching any decently performant WebGL techniques that could improve rendering quality by more realistically lighting a scene. A few of the ones I found are listed above. PBR materials could definitely be the avenue for defining a reflectivity property some techniques would use; also a possibility to default some small basic reflectivity property to make lighting more realistic in general. Please provide any feedback you have as well.

[mgooding]

I don't think web client path tracing is a viable option for our user's models. This is a pretty good implementation, and the performance is not great even on simple models with powerful hardware: https://github.com/gkjohnson/three-gpu-pathtracer

[pmconne]

@markschlosseratbentley I don't think we should build further enhancements atop our current hacked-together lighting system. PBR has become the standard for realistic lighting - if we want more realistic lighting, we should implement PBR first. That also would immediately improve our compatibility with other data sources (glTF, 3d tiles, DGN materials, Revit, etc).

[mgooding]

Agree with @pmconne , but that doesn't need to block this project for @eringram . I'd suggest researching and prototyping in Three.js or any WebGL framework that has PBR implemented already. You can use https://developer.bentley.com/apis/mesh-export/ to create glTF files from iModels to get representative test data (though keep in mind this is a static rather than HLOD format, so you need to pick smaller iModels or export a subset of the iModel).

[markschlosseratbentley]

@pmconne @mgooding Thank you both for your feedback. @eringram Matt's suggestion above sounds good to me - this would become more of a project to evaluate what aspects of PBR are most readily able to be implemented to iTwin.js.

[markschlosseratbentley]

TODO:

• [x] Export a sampling of real iModels to glTF.
• [ ] Render using effects in Three.js / standalone WebGL examples (SSR effect, etc.)
• [ ] Decide if the effects result in a good rendering of the variety of datasets.
• [ ] Consider implementing the best effects in iTwin.js as an implementation of the relevant PBR property.