three.js
three.js copied to clipboard
mrdoob
Add shadow map support for RectAreaLights (brainstorming, R&D)
Description of the problem
It would be very useful for realism to support shadows on RectAreaLights.
I am unsure of the best technique to use here as I have not researched it yet beyond some quick Google searches. I am not yet sure what is the accepted best practice in the industry?
Two simple techniques I can think of:
- one could place a PointLightShadowMap at the center of a rect area light and it would sort of work.
- less accurate, one could place a SpotLightShadowMap with a fairly high FOV (upwards of 120 deg, but less than 180 deg as that would cause it to fail) at the center of the rect area light and point it in the light direction.
(I believe with the spot light shadow map you may be able to get better results for large area lights if you moved the shadow map behind the area light surface so that the front near clip plane in the shadow map frustum was roughly the side of the area light as it passed through the area light plane. I believe I read this in some paper once, but I can remember the source of it.)
/ping @abelnation if you have any ideas as you are the current resident Area Light expert. :)
I refactored the area light code and I am familiar with it, too. For shadow support, see https://eheitzresearch.wordpress.com/705-2/.
For shadow support, see https://eheitzresearch.wordpress.com/705-2/.
Interesting. I think that may be a bit too computational intensive for WebGL at this time though as it makes an assumption that ray tracing is hard ware accelerated. It is likely years before that comes to WebGL unfortunately.
I've of the opinion that while my two suggestion (point or spot light shadows) are not physically accurate when it comes to soft shadows, they are better than no shadow and are both very fast (as fast as our other shadows at least) and easy to implement (as the code already exists.)
@bhouston Oh, yeah... there is also https://github.com/mrdoob/three.js/pull/13057.
If you read the entire discussion, I think the consensus was to add 'cookie' support to SpotLight, which would presumably allow for a rectangular shadow.
NVidia's Percentage Closer Soft Shadows do a decent job of approximating area light shadows. I was toying with hacking them in to a THREE.js scene here: https://gkjohnson.github.io/threejs-sandbox/pcss/index.html. It's using a directional light shadow map, at the moment, but a spotLight one might be better.
They're not perfect solution, though -- they have some artifacts when shadow edges get closer to each other and require a lot of samples to get a really smooth shadow.
There is already a pcss implementation in three.js that myself and oreshant wrote a while back.
Best regards, Ben Houston http://Clara.io Online 3d modeling and rendering
On Sat, Jul 21, 2018, 11:02 PM Garrett Johnson [email protected] wrote:
NVidia's Percentage Closer Soft Shadows http://developer.download.nvidia.com/shaderlibrary/docs/shadow_PCSS.pdf do a decent job of approximating area light shadows. I was toying with hacking them in to a THREE.js scene here: https://gkjohnson.github.io/threejs-sandbox/pcss/index.html. It's using a directional light shadow map, at the moment, but a spotLight one might be better.
They're not perfect solution, though -- they have some artifacts when shadow edges get closer to each other and require a lot of samples to get a really smooth shadow.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mrdoob/three.js/issues/14161#issuecomment-406837539, or mute the thread https://github.com/notifications/unsubscribe-auth/AAj6_RuFJPv7y5DbEfJnDu26DB3cD2bhks5uI-tJgaJpZM4UQODd .
There is already a pcss implementation in three.js that myself and oreshant wrote a while back.
https://threejs.org/examples/#webgl_shadowmap_pcss
This runs with nice 60 FPS on my iMac^^.
There is already a pcss implementation in three.js that myself and oreshant wrote a while back.
I hadn't seen that! It looks good. I didn't know you could generate poisson disk-like values that way -- it looks like it has a nicer distribution and gives better results than the way I was using. I may borrow that if you don't mind!
1 FPS on my Pixel 😢
Ha! It wasn't working on my phone, either. I didn't optimize it at all and there's a lot of work going on in loops that could be moved out.
@bhouston's implementation is pretty fast, though. Maybe there's some variant of this that can be used for area light shadows? It would be nice to have any kind of shadow support for area lights so this type of implementation could be more easily plugged in.
NVidia's Percentage Closer Soft Shadows do a decent job of approximating area light shadows. I was toying with hacking them in to a THREE.js scene here: https://gkjohnson.github.io/threejs-sandbox/pcss/index.html. It's using a directional light shadow map, at the moment, but a spotLight one might be better.
They're not perfect solution, though -- they have some artifacts when shadow edges get closer to each other and require a lot of samples to get a really smooth shadow.
Nice! It's not perfect but it looks better than no shadow... I get 30fps on my Asus ROG Phone 2. And 45fps on my integrated graphics laptop.
Here is an example of raytraced shadows in three.js only the shadow component is being shown in these videos.
60+fps with 1 shadow sample per pixel... With some smart filtering the results could look pretty good.
https://youtu.be/O21mKUtMtSg https://youtu.be/2Tn93hf9kNw
This could be used to help with rect area light shadows.
A live web demo is in the works.
Related #13908 Improving Shadows #14048 Soft Shadows via Light Source Sampling #14051 Adopting a Progressive Photorealistic Global Illumination in Three.JS
Related issue: I don't know how to implement some sort of shadows with the current IBL code.
Say that I want to add a DirectLight in the car example but ignore the actual light source and only use the shadow map to affect the IBL code.
I've been playing around with several ray traced effects.
- Shadows - from direct light sources, including rect lights
- Ambient Oclusion
- Global Illumination
Of the 3 I think per-pixel GI which would work with IBL is the hardest to achieve real time frame rates.
However, per-vertex GI and might be possible at good frame rates. The ray tracing code is being written with the idea that vertex and fragment shaders will be able to use it.
@mrdoob, @WestLangley, What space are the uniforms "pointLights[].position" in??? I'm doing the ray-tracing in world space but the shadows don't seem to match up.
Ok, after looking into it... It looks like it is in model view space... Is there a good way to get the light positions in world space?
Modify WebGLRender to send in both world and view space?
Ok, I got it working with a point light... now to test rect area lights!
Is there a way to just jitter lights sources within the emission volume and accumulate in order to get soft shadows in something like 32 renders that you just accumulate?
I view this as something that works well with the sub-pixel camera jitter that I accumulated in the TAA Three.js example.
On Sat, Dec 21, 2019 at 10:04 PM Samuel Sylvester [email protected] wrote:
Ok, I got it working with a point light... now to test rect area lights!
[image: image] https://user-images.githubusercontent.com/10963749/71316445-4b18bb00-2435-11ea-94d9-c2f0f4116097.png
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mrdoob/three.js/issues/14161?email_source=notifications&email_token=AAEPV7MQQB4SDPHJAOA5YXTQZ3KLVA5CNFSM4FCA4DO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHPHXHQ#issuecomment-568228766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEPV7JKIZ62J3YXCVVYHVDQZ3KLVANCNFSM4FCA4DOQ .
--
Ben HoustonCTO *M: *+1-613-762-4113 [email protected] Ottawa, ON
[image: threekitlogo1568384278.png] https://www.threekit.com/ Create a better visual customer experience with Threekit https://www.threekit.com/
[image: Learn how to scale product visuals with your growing business] http://signatures.threekit.com/uc/5d0cfd47860b6864aee40938/c_5da8cc74c90a0e0043fe0323/b_5dade3e4e85fae00caea3d76
To be clear I think the way to jitter in a rect light is to have a single point light that you move along the emission surface. You need a good sampling method like Poisson Disk or something, https://www.google.com/search?q=Poisson+Disk. And then the resulting shadows should be soft and appropriate for a rect light.
Although I have been confused on the FOV one should set on the point light, and maybe you should always point it in the normal direction from the rect area surface? It is these minor details that have keep me from actually going forward with an implementation of accumulative sampled emission source-based shadows.
On Mon, Dec 23, 2019 at 9:00 AM Ben Houston [email protected] wrote:
Is there a way to just jitter lights sources within the emission volume and accumulate in order to get soft shadows in something like 32 renders that you just accumulate?
I view this as something that works well with the sub-pixel camera jitter that I accumulated in the TAA Three.js example.
On Sat, Dec 21, 2019 at 10:04 PM Samuel Sylvester < [email protected]> wrote:
Ok, I got it working with a point light... now to test rect area lights!
[image: image] https://user-images.githubusercontent.com/10963749/71316445-4b18bb00-2435-11ea-94d9-c2f0f4116097.png
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mrdoob/three.js/issues/14161?email_source=notifications&email_token=AAEPV7MQQB4SDPHJAOA5YXTQZ3KLVA5CNFSM4FCA4DO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHPHXHQ#issuecomment-568228766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEPV7JKIZ62J3YXCVVYHVDQZ3KLVANCNFSM4FCA4DOQ .
--
Ben HoustonCTO *M: *+1-613-762-4113 [email protected] Ottawa, ON
[image: threekitlogo1568384278.png] https://www.threekit.com/ Create a better visual customer experience with Threekit https://www.threekit.com/
[image: Learn how to scale product visuals with your growing business] http://signatures.threekit.com/uc/5d0cfd47860b6864aee40938/c_5da8cc74c90a0e0043fe0323/b_5dade3e4e85fae00caea3d76
--
Ben HoustonCTO *M: *+1-613-762-4113 [email protected] Ottawa, ON
[image: threekitlogo1568384278.png] https://www.threekit.com/ Create a better visual customer experience with Threekit https://www.threekit.com/
[image: Learn how to scale product visuals with your growing business] http://signatures.threekit.com/uc/5d0cfd47860b6864aee40938/c_5da8cc74c90a0e0043fe0323/b_5dade3e4e85fae00caea3d76
@bhouston I think https://github.com/mrdoob/three.js/issues/14048 is what you are talking about.
I liked the demo with the twisted knot lighting source . I guess the problem that I saw is that the reflection of the light source never resolved to the correct image.
One plus for the ray-traced shadows is that It be used to handle transparent shadows. It can also be used for Global Illumination and many other lighting effects.
@bhouston https://github.com/bhouston I think #14048 https://github.com/mrdoob/three.js/issues/14048 is what you are talking about.
Sorry, I tend to forget what I previously suggested, but at least I am consistent.
I liked the demo with the twisted knot lighting source . I guess the problem that I saw is that the reflection of the light source never resolved to the correct image.
It is because he isn't doing random sampling but an exhaustive sampling method -- he is also doing a much harder light source than just a rectangular area. His method would be slow accumulative with rasterization rendering or ray tracing -- it is the sample method that was wrong, not the technique. Also you can multiple light samples per render if you are so inclined. I think that 32 to 256 renders on the high end is needed, depending on the light type and the number of light samples per render. At 60 fps is this 1 - 4 seconds. I think of it as a simplified "instant radiosity" solution.
You are right that ray tracing is amazing. I just thought that jittering lights is easier than redoing the whole lighting system, and then you also have a good solution for real-time, the shadows just are hard and then on stop they soften to perfection, and the render algorithm doesn't switch, just things jitter and accumulate.
On Mon, Dec 23, 2019 at 11:03 AM Samuel Sylvester [email protected] wrote:
@bhouston https://github.com/bhouston I think #14048 https://github.com/mrdoob/three.js/issues/14048 is what you are talking about.
I liked the demo with the twisted knot lighting source . I guess the problem that I saw is that the reflection of the light source never resolved to the correct image.
One plus for the ray-traced shadows is that It be used to handle transparent shadows. It can also be used for Global Illumination and many other lighting effects.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mrdoob/three.js/issues/14161?email_source=notifications&email_token=AAEPV7L4ZVKQJILFWZQ6JE3Q2DOLPA5CNFSM4FCA4DO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHRNA4I#issuecomment-568512625, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEPV7LXP5CSOMBZF2BMNR3Q2DOLPANCNFSM4FCA4DOQ .
--
Ben HoustonCTO *M: *+1-613-762-4113 [email protected] Ottawa, ON
[image: threekitlogo1568384278.png] https://www.threekit.com/ Create a better visual customer experience with Threekit https://www.threekit.com/
[image: Learn how to scale product visuals with your growing business] http://signatures.threekit.com/uc/5d0cfd47860b6864aee40938/c_5da8cc74c90a0e0043fe0323/b_5dade3e4e85fae00caea3d76
@bhouston I like solution proposed in #14048... especially for simple geometries like a rect light. It looks like that method should be pretty easy to employ. Is there a fork of three that has this solution?
Regarding the ray tracing... On desktop class hardware the ray tracing solution works in real-time at 1spp. See... https://youtu.be/amX3icmbpzY.
I have not yet tested mobile.
In the demos that I've been working on, the lighting term is calculated using existing code but the shadow is ray traced instead of shadow mapping.
@mrdoob, @bhouston So, I updated the demo to reflect Nvidia's rtx demo from GDC 2018
And I have it running real-time on my laptop and on my cellphone...
Laptop demo... https://youtu.be/MKCKXRVks3I Cellphone demo... https://youtu.be/P-cnT2hYNF4
I also improved the distribution of the shadow samples to improve shadow realism
@mrdoob, @WestLangley , @bhouston and others
I got shadows for rect area lights working with raytracing.
The demo shown here... https://youtu.be/tZmlb29OUBU
Has point lights and a rect area light.
Some improvements to the distribution of shadow samples are in the works. Also, it looks like for area lights 2 shadows will need to be calculated for each light. A diffuse shadow and a specular shadow.
At present only the diffuse shadow is being calculated.
I plan on setting up a live website with this demo and also pushing these changes to my fork of three soon
That's looking great! Does it only use a single depth texture too?
@mrdoob thanks! It actually is proper ray-tracing, currently it uses 2 textures. One stores world space geometry and the other has the BVH data.
With this setup we could calculate RTX ambient oclusion and with a few updates GI would be possible too!
To run the demo in browser see... https://three-rtx.azurewebsites.net/
Note... This is ray-traced and I found that my integrated GPU could only handle one light source with ray-traced shadows with decent frame rates. But when I enabled my RTX 2080 I could have 2 light sources ray-traced with 16spp per light at twice the frame rate. That's a little less than 64x faster on a discrete GPU!
I also tried running this on my phone and I found that you are going to want a 2019 flag ship or better for 30+ fps at 1080p.
I believe there are a few more improvements to squeeze out more performance and less noise.
I plan on pushing the code I have now to my fork and then continuing to make tweaks.
Note: I have one tweak that improves perf by over 50% average fps... it works on the shaders first compile but if a second compile is triggered then funny things start to happen... It looks like its related to my use of the pre-processor