Refractions different in each eye for VR

[daleeidd]

Describe the bug Refractions appear slightly different in each eye.

Screenshots / video

BRP

HDRP

URP. I believe the black highlighted in yellow box is the occlusion mask.

Versions BIRP, URP and HDRP

To Reproduce

1. You'll need an object underwater
2. Reduce depth fog density to zero to make it easier to see
3. Increase refraction to make issue easier to see

Platform

• Editor
• Windows

I've only tested on the above.

Hardware 2080 Ti

Additional context The issue scales with the refraction value. So it isn't that noticeable when refraction is at lower values. Even if minor it could contribute to VR sickness.

[moosichu]

Isn't that just the way reflactions/refractions work when you have a waves on a water surface. Each eye is going to receive light reflected from diferent locations.

https://www.youtube.com/watch?v=G22nvIbe4-M

[daleeidd]

I see. Considering me now informed. Thank you.

This is one of the more extreme ones I have when refractions are set to the maximum of 2. I'll check this again at 1. And I will double check to see if the black appears in the HMD. Pretty certain it is the occlusion mask (or some strange value). The underwater object is the same white sphere as the others.

[huwb]

There are things that are normal in the real world that are very disturbing in VR. One example is specular highlights from a light source, if they are present in one eye and missing from the other it glimmers and looks bad in VR, even though it's physical.

There might be something we can do to unify the refraction across the two eyes. Or I guess just reducing refraction strength is a kind of fix?

[moosichu]

There are things that are normal in the real world that are very disturbing in VR. One example is specular highlights from a light source, if they are present in one eye and missing from the other it glimmers and looks bad in VR, even though it's physical.

There might be something we can do to unify the refraction across the two eyes. Or I guess just reducing refraction strength is a kind of fix?

What's the reason for this? Is it because things in VR lack different focal depths so everything is constantly in-focus?

[daleeidd]

What's the reason for this? Is it because things in VR lack different focal depths so everything is constantly in-focus?

Most likely. Our brain does processing of what we see (perception correct, colour compensation etc). Perhaps without depth it cannot do that properly. There is a lot of information missing from a finite medium. So relying on the physics of lenses alone might be incorrect. No idea if the brain does it for refractions.

Or I guess just reducing refraction strength is a kind of fix?

Sorry. I will check again tomorrow. Been taking a break to learn some fundamentals I was missing.

[daleeidd]

I have found that refractions are much worse underwater in VR:

BIRP with refractions at ~0.4

I found that a refraction value of 0.2 gave best results across pipelines. Underwater there was still some double vision around the periphery but I doubt anyone would notice.

BIRP didn't seem too bad going up to a refraction value of 1 above water.

~~URP appears to be the worse affected. Especially going above one where these dark patches would appear as shown in screenshots. It isn't the occlusion mask as I thought it might be, but I don't know what it is.~~ It is.

HDRP fairs the best overall. I found that I could go all the way to 2 and the difference in each eye isn't noticeable (I couldn't test underwater).

[furkandinc]

I went through this issue couple of months ago. I didnt have this issue with multi-pass rendering. I think the problem in underwater refraction with single pass rendering happens because of double wide render texture usage in single pass rendering. I got this weird effect at corners underwater in VR. I guess refracting with high refraftion value will take background far away from the fragment point. While green fragments were refracted well, red fragments were making weird effects. Due to this effect it is quite uncomfortable to look from underwater. To solve this issue added some boundary in shader to clip refraction points. End result cant be same because of the eyes are in different world positions. Result is like this. If I clip too much, background objects were extended too much in refraction. It is not a generic solution. There should be a better solution. But it is ok for me.

.

[moosichu]

Yeah, this is a hard one for @huwb and myself to work on and fix because a big part of the problem is how this actually looks in stereo. @daleeidd are you happy deciding on an effective solution to this, as you are really the only one able to effectively evaluate what works and doesn't. One optoin could be to add some VR guidelines to the README?

[daleeidd]

@furkandinc Thank you for your solution. I made a test branch for built-in if you would like to try and give your feedback on: fix/vr-occlusion-mask-refractions

The exposed properties on the material won't be kept but rather were used to find a nice value to clip UVs. There is also a fallback where if the occlusion mask is sampled, then alpha 0 zero is returned and we just resample with a much smaller refraction value. Unfortunately, the fallback won't work in URP since it takes its opaqueness very seriously and never returns alpha 0. I will have to report this to Unity.

I wasn't able to find any better solution at this stage.

[furkandinc]

@daleeidd I have tried your work. With the default clip values you placed, it looks great in single-pass vr. And there is the same issue with multi-pass rendering. I made a mistake while commenting my last comment that I didn't get problem with multi-pass. I was using slightly modified Crest and I commented it with my modified version. Sorry for rework :) I guess same technique can be used for multi-pass rendering.

[daleeidd]

@furkandinc Thanks. I'll get that fixed up.

It also appears that URP now returns alpha zero when the mask is sampled so the fallback will also work.

Will do a PR soon.