KhronosGroup
FB_materials_modmap
A draft pass at implementing a vendor extension for lightmap functionality
I haven't looked at either in great detail, but since they both have lightmap in the name, what are the differences between this extension and https://github.com/KhronosGroup/glTF/pull/1017?
Thanks @idorurez!
Because this is a draft, I'll respond below on the assumption that FB is not yet using this extension, and is looking for either feedback on the approach or alignment with other vendors. If that isn't the case — e.g. the extension is already in use or locked in — and you'd simply prefer for this to be merged as-is, we can also make that happen.
This does not define how the lightmap will be applied, just that one exists.
For a multi-vendor or Khronos extension, I think we would need to go farther than just identifying the texture as a lightmap. Ideally we would define how it will be applied, assuming some consistency in client applications is possible. If that isn't possible, we would at least want to understand the spectrum of implementations that exist, and provide some documentation and guidance for implementers. For occlusionTexture, as an example, pre-existing implementations have quite a few differences.
- Unity: See Lightmaps: Technical Information. Offers several quality settings, with the "normal" setting being RGBM.
- Unreal: Haven't found much. One link suggested lightmaps are greyscale in Unreal?
- threejs*: RGB channels are multiplied against lightmapIntensity, then added to indirect diffuse. Alpha channel is ignored.
^Not a lot of detail, hopefully others can help improve this! :)
Stepping back from lightmaps for a moment, there are several variations of lighting that either exist in glTF, or have been suggested at some point:
- [x] punctual lights
- [x] ambient occlusion maps
- [ ] ~~ambient lights~~ (not pursuing)
- [ ] area lights
- [ ] lightmaps
- [ ] spherical harmonics
- [ ] light probes / irradiance probes
- [ ] IBL
Looking at this list, I'm really not confident that all of these things belong in the glTF format... Understandably users have different needs; some want efficient solutions for mobile devices today, others want higher quality or more future-proof features. But all the same, if we try to do everything the format will get complex very fast.
Of the not-yet-supported features on that list, where do we think glTF should be — say, 2-5 years out?
/cc @bhouston
I'd like to see all of those lighting features supported in glTF, although I think most of them should be implemented as extensions. I think that discussion should be moved into another issue.
As for lightmaps, I'd like to see a multi-vendor (EXT) supported lightmap extension, specifically one that was supported in Unity, Unreal, Babylon, and Three.js This one looks good as-is and I would imagine adding support for Three.js with a custom shader that used multiplicative lightmaps.
Without getting too detailed (or do we need to?), should we define the limits on the encoded values based on the encoding method and color space?
I'm referring to the Unity Technical Documentation on Lightmaps, where they mention various clamped values for RGBM and dLDR encoding.
Maybe it's worth keeping it to a single encoding method to keep things simple.
In three.js parlance (which is based on Frostbite's), lightmaps should be considered indirect diffuse - search for "indirect diffuse" in this PDF:
https://seblagarde.files.wordpress.com/2015/07/course_notes_moving_frostbite_to_pbr_v32.pdf
The implementation that put them into indirect diffuse was this large PR here:
https://github.com/mrdoob/three.js/pull/7324/files#diff-29f010d669a36f5263d5351f4702e64fR62
Although we should upgrade the texture2D( lightMap, vUv2 ).xyz to be lightMapTexelToLinear( lightMap ).rgb so that encoded (e.g. HDR) textures can be supported with ease.
It was discussed how lightMap should interact with other lights (should it occlude other lights?) here:
https://github.com/mrdoob/three.js/issues/7377#issuecomment-149366585
lightMaps to be physical should be specified in units at least once combined with the lightMapFactor - and it should be illuminance in lux.
The reason we put light maps into the indirect diffuse component is that it allows us to consider the indirect diffuse component correctly in the material's BRDF rather than just using a simplified material model of just (diffuse * lightMap) for lightmaps.
We're getting way into the weeds here, as Three.js has a much involved and proper model for a physically based implementation of lightmaps.
Again, without giving a whole lot away, there is specificity on the unlit texture of what we're expecting to multiply against, so I was considering KHR_materials_unlit as a requirement.
However, @bhouston makes a good point: Don't call it lightmap. Name it something like FB_materials_modulationMap to avoid confusion on implementation, much like the thread he has linked. I think this is a much better nomenclature because it implies it's application.
Coincidentally, I called this FB_materials_multMap before refactoring it as FB_materials_lightmap.
What does everyone think?
On one hand this is "way into the weeds" for a vendor extension that is already sufficient for the vendor's needs. But it's definitely a discussion I would want to have in detail before an EXT_ or KHR_ extension. If threejs has a physically-based lightmap model now, I don't imagine we would implement the multiplicative version, although certainly threejs users can easily do this using custom shaders or THREE.NodeStandardMaterial.
From my perspective we can merge this (as a vendor extension) any time; it is just a public description of what Facebook needed and built. I agree that another name might be better, if that's ok.
Before an EXT_ or KHR_ extension that other tools would implement, I do think we should have a sense for how/if it fits our roadmap (https://github.com/KhronosGroup/glTF/pull/1658#issuecomment-523091780) and, if we decide to proceed, agree on a physical model.
Thanks Don, your advice and help is appreciated. I'm going to rename this as appropriately after consulting with a few engineers here to be sure everyone's happy and update this pull request.
Where are we with this? Are lightmaps fully supported with clearcoat and metalness ? I'm having some issues with configuration
Curious why this extension never made it. Is this the same as EPIC_Lightmap extension for gltf?
Note that this extension does not add lightmaps to glTF — it rather adds a broadly specified mathematical operation. It sounds like that structure was useful to Facebook in an (unspecified) project for lighting-related customization, but given the discussion under https://github.com/KhronosGroup/glTF/pull/1658#discussion_r316149184, this is not the right approach for a standardized lightmap extension.
Because this was a proposal for a vendor extension, I think it should be merged without further delay if the vendor is still interested in having the extension specification hosted here. If not, the PR can be closed. In either case, design for a lightmap extension probably needs a new PR, with volunteers to work on its definition.