specular_weight reinterpretation

[portsmouth]

It was noted on Slack by Masuo that it seems a bit strange that the refraction changes when specular_weight is varied. This happens because in the current spec, we say that the specular_weight controls the IOR of the dielectric.

For a glass object, this may be unintuitive:

specular_weight 1	specular_weight 0.1

The issue is that the name implies "specular presence", so in this case the artist probably expects the control to merely kill the reflection from the glass, not change the refraction as well. We do already have a control that does that, namely specular_color, but it is unintuitive to use that control for dialing the dielectric reflectivity.

So the proposal here is to have the specular_weight simply function as a multiplier to that existing color (so it becomes simply an overall weight factor for the whole lobe). This matches how the specular_weight functioned in Standard Surface.

The change to the spec amounts to a wording change only:

Note that this change requires:

• No change to the existing parameter names/ranges.
• Some other minor textual changes in the spec, as in the PR
• A small change to the existing implementations (a simplification in fact).

[portsmouth]

Note that this simplifies the presentation in the spec (and the implementations) somewhat:

We originally went with the approach of specular_weight controlling the IOR to keep everything physically-based, and to align with the Adobe model which has the "specular IOR level" control for texturable IOR modulation.

With this change the ability to modulate IOR directly, with this [0, 1] control is lost. However arguably what users are looking for in the control is more like the uniform suppression of the entire F, like a lobe "weight" (allowing it to be totally killed off), not just suppressing F0 while keeping the grazing highlights intact. The use of textured specular_weight to produce variable specularity should be discouraged anyway, in favor of textured roughness, for better realism.

For the metal lobe, specular_weight already does this uniform suppression of the whole F. (We needed that in order to be able to kill the lobe). Also the specular_color already functions as an unphysical weight for the dielectric, so it does not really impact the physical basis of the model to have specular_weight simply multiply that.

The need to genuinely texture the IOR (e.g. for representing clothing with gems, in a single material) seems more like a niche, unusual use case (that would normally be done simply by modeling the gems separately). And it seems sub-optimal to design the parametrization around constraints of the DCC (i.e. that in the niche case where IOR has to be textured, it can't simply be controlled via [0,1] float textures by plugging in nodes which remap IOR in the range [1, 3]).

Also note that this was the behavior in Standard Surface which has been fairly battle tested, so we can be reasonably confident it is fine, at least for users happy with Standard Surface. (Although Standard Surface did not explicitly allow the weight to exceed 1).

[fpsunflower]

An alternative take could be to split the IOR used for fresnel from the IOR used for refraction (ray bending). This is the approach we took in Unreal's path tracer and it does have a number of advantages:

• it's closer to what artists expect. They have independent controls over ray bending vs reflectivity
• it does not exclude the physically correct behavior (having both IORs match) - it is strictly a superset that is more expressive
• in the context of rough refraction, reducing the refraction IOR is a way to lower the refraction roughness without touching the reflection roughness (the specular highlight). The way the microfacet math works out, lower IOR means less bending which means roughness does less.

One natural reservation about this idea is -- how does this work with total internal reflection? The refraction equation for computing how the ray bends falls apart at a critical angle -- if this angle doesn't match the fresnel curve, one would think you would get a weird artifact if they don't line up.

There is a property of the dielectric fresnel equations that isn't immediately obvious, but allows this to work:

  Fresnel(cos_theta, 1 / ior) == Fr(cos_theta_t, ior)

Here cos_theta_t is the refraction angle cosine: Sqrt[(1-(1-cos_theta^2)*ior^2] and ior is the refraction index (assume to be >1 here so we are clear about which side is which).

In plain english, what this means is that the Fresnel curve you get when entering the material, is the same as the compressed Fresnel curve you get when exiting the material. As long as you do the "compression" using the refraction IOR, you are free to use any fresnel curve you want, you only need a curve with the right F0 value and have it tend towards 1.0 at grazing angles. Past the grazing angle when leaving the material, you have TIR and just use 1.0 (full reflection).

This also gives a recipe to replace the dielectric fresnel with the Schlick approximation if you want to (as has been documented in other talks like the Disney BSDF writeup or Natty Hoffman's talk on Fresnel).

Some other details that might be relevant -- we use the albedo scaling approach for energy conservation. At first I was worried that decoupling the IORs would require an extra dimension in the albedo table. However in practice, the specular IOR has a minimal effect. So sticking to a 3D table with (cos_theta, roughness, refraction IOR) is sufficient for good results.

I don't recall if the OpenPBR spec spells out the behavior of nested dielectrics -- but this can all be taken into account easily if you want. Again, you have a additional degrees of freedom over the behavior here, but the physically correct behavior can be kept.

[portsmouth]

This decoupling of the ray bending and the reflection seems like a more aggressive "non-physical" modification than having specular_weight be a simple multiplier. At this point you are basically just altering the Fresnel formulas to taste artistically.

If we did this, then specular_weight is no longer behaving in a "physically correct" way, which I thought was the argument for having it control the IOR. (To get the correct physical behavior with the proposed decoupling, artists have to not change specular_weight from the default).

If it's considered reasonable to have this non-physical behavior of specular_weight (altering the Fresnel formulas for the R and T modes separately) I find having it be a scale factor of the Fresnel easier to deal with, than thinking about it as decoupled IORs.

[portsmouth]

It would be good to understand in more detail the decoupled IOR proposal. This is breaking physics, so it's not easy to reason about how it should behave in full detail.

I take it the IOR ratio seen by light rays is made different according to the specific scatter configuration (which is non-physical, but mathematically possible). Given the interface with an exterior ("above") and interior ("below"), there are 4 (single) scatter modes to consider:

• light incident from above reflects (R+)
• light incident from above refracts (T+)
• light incident from below reflects (R-)
• light incident from below refracts (T-)

(There are more modes if you consider microfacet multiple scattering, but can ignore that for now. Or maybe we consider these modes to be macroscopic only, I'm not sure).

The proposal I assume will have both the "from above" modes see (the same) modified IOR ratio, which potentially differs from the "from below" modes which both see the unmodified original IOR ratio. (The ratio seen from above will be the one we currently calculate for the whole interface, via the specular weight).

In the case where the interior IOR is higher, light incident from below can undergo TIR. So the discrepancy I take it will be that light from below that undergoes TIR, may also correspond to a refraction from above (which is physically impossible)? I'm not sure what artifact this causes.

In the discussion above about "stretching" the Fresnel curve, I'm unclear how this works, since the Fresnel coefficient will be determined by the IOR ratio seen by the mode. Maybe some diagrams could help..

I generally worry that by breaking the standard physical picture, we could make it quite tricky to understand how to actually implement this in a sensible way, given the rest of the complexity of the microfacet physics being accounted for (multiple scattering, nested dielectrics, shadow masking etc.).

[fpsunflower]

It might be helpful to do a demo of an implementation of this, I can try to have a setup of this ready for the next OpenPBR meeting so we can noodle the parameters interactively and show how it works.

I am not sure I exactly understand your questions, but I do agree that its a bit subtle and I had similar reservations as you before I worked through it. Once you see the small changes required in code to make this work, you realize that this is actually a fairly minimal change. If you have an implementation that uses the Schlick approximation in a microfacet roughness model, you've actually already done the hard work. That might be the best place to approach the implementation. You can switch back to the real fresnel equations afterwards if you prefer them. In your microfacet code, you only need to deal with the ray bending IOR for all microfacet half-vector math. The only thing you need to mess with is the implementation of the F term. Instead of taking in cos_theta and eta as arguments, you should provide cos_theta, bending_ior and specular_ior. When entering a material, you just evaluate the fresnel math directly with cos_theta and specular_ior, there's no TIR to worry about. When exiting a material, you first evaluate the "squeezed" refraction angle (and TIR) with the bending_ior and as long as there is no TIR, you evaluate your fresnel curve using the squeezed angle and your specular_ior value. The identity I showed above is helpful because it shows that this is equivalent to evaluating the fresnel math "directly" from the raw equations.

Maybe a blog post working through the details would be helpful?

[portsmouth]

It might be helpful to do a demo of an implementation of this, I can try to have a setup of this ready for the next OpenPBR meeting so we can noodle the parameters interactively and show how it works.

Maybe a blog post working through the details would be helpful?

Those would be excellent, thanks @fpsunflower!

It would be good to make sure we examine the most general case, so arbitrary ambient IOR (i.e. assume medium tracking). Note also that the dielectric may be under the coat, which has an arbitrary IOR as well.

(In nested dielectrics, I recall there is in general no "interior" and "exterior", there are just adjacent dielectrics with different IOR, and the Fresnel (for rays incident from either side) depends on the ratio. Now we are hacking this somehow, in some sense that needs to be clear).

[fpsunflower]

The discussion of nested behavior is largely orthogonal to all this. The Fresnel equations can be written in a few different ways. The most literal, straight from the textbook form looks like it needs two IORs (inside and outside). In reality, only the ratio between those is important like you said. Likewise for Snell's law for the ray bending.

In my description above, when I used bending_ior and specular_ior I should have made it more clear these should be the ratio between internal and external IORs. When the ratio is >1 you are "entering" and when <1 you are "exiting" regardless of what the geometry looks like. Like I mentioned above, the most important thing to realize is that the Fresnel equation when exiting is the same as entering just squeezed (with a clamp to 1.0 as soon as you are past Brewster's angle).

You are right that we should spell out exactly what the interaction with the coating should be. As long as you take the proper ratio for both quantities, you can preserve the physical behavior. You also gain additional flexibility if you want it, but we don't need to reinvent the wheel. Whatever language we already have in the spec could just be extended to cover both IOR ratios.

Another area that will need to be updated is the section on dispersion. I have to admit I haven't implemented this part in a decoupled setting myself, but I suspect that again we could just view the IOR tweak that dispersion implies as a tweak to both the specular IOR and the bending IOR.

[portsmouth]

Would it not work to simply have the reflection from above (i.e. from the exterior) compute the Fresnel factor using the modulated IOR, with everything else unmodified? That includes unmodified reflection from below (i.e. from the interior), so TIR is unaffected. Also, completely unmodified refraction directions and transmission coefficients (that breaks energy conservation technically, but it is the price for the artistic hack).

This is a rather similar, relatively harmless simple hack, to the existing specular_color multiplier. It only changes that one Fresnel factor, except in a somewhat more physically acceptable way (maintaining the white at grazing). It only changes the reflection seen from outside (as should specular_color) but presumably that is what we (I.e. the artists) care about when tweaking specular_weight.

For reference we said:

The specular_color parameter modulates the Fresnel factor of fdielectric, but only for the initial reflection of light incident from above, while the light transmitted from above or below (or reflected from below) is assumed to be unaffected. This is technically unphysical if altered from the default white color (as real dielectrics have a Fresnel factor dependent only on the index of refraction), but can be useful in practice to artificially tint the specular highlight without disturbing other aspects of the light transport, i.e. the reflection due to scattering from the internal medium, or the reflection from below, or the transmission from above or below.

We would just amend that to say that the specular_weight functions via this same modulation, by changing the IOR ratio in that one Fresnel factor. (It is then further multiplied by specular_color). Everything else is the same and just uses the unmodified IOR.

That way it avoids this quite confusing notion of there being 2 IOR ratios involved. We are simply calculating the reflection Fresnel factor in a modified way. Dispersion (refractive) for example is not affected, since we define that the only modification to light transport is that one Fresnel factor.

(This is so simple to do, I will implement the spec change in the next commit, for discussion).

———

EDIT:

There is the case where the interior IOR < exterior IOR to consider. In that case, the Fresnel reflection from above has TIR for sufficiently grazing angles. The critical angle will change as the specular_weight changes. I don’t see this causing any particular issue though.

Note also the coat may have higher IOR than the interior. In that case we mentioned that implementations may want to invert the IOR ratio (in the specular dielectric BSDF) as a trick to prevent the TIR (essentially faking the effect of the refraction in the coat). This trick would still apply.

[peterkutz]

The Disney BSDF notes @fpsunflower mentioned are from the 2015 Physically Based Shading course and linked from here: https://blog.selfshadow.com/publications/s2015-shading-course/#course_content

Attached is the relevant part of the course notes. While it was a long time ago, I was the one at Disney who observed and implemented this scaling of the Fresnel curves, so feel free to let me know if you have any questions about this.

[peterkutz]

Last year, @portsmouth and I also discussed a similar concept in the context of polarization. Here's a recap of that, including diagrams illustrating the scaling.

Like @fpsunflower mentioned, F(n_t / n_i, cos(θ_i)) = F(n_i / n_t, cos(θ_o)) due to reciprocity. In other words, if you swap the IORs and transform the angle using Snell's Law, the reflection and transmission coefficients remain the same.

You can try it in an online Fresnel calculator like this one: https://www.lasercalculator.com/fresnel-reflection-and-transmission-calculator/

You can also see in a chart that swapping the IORs only compresses the curves into a smaller set of angles (nonlinearly with respect to angle). You can try it here: https://webs.optics.arizona.edu/gsmith/FresnelCalculator.html

And below are example curves from that last link.

Going from IOR 1 to IOR 1.5:

Going from IOR 1.5 to IOR 1:

[peterkutz]

Here's another example, plotted in Mathematica, illustrating that the Fresnel curve for going from high to low IOR can be transformed to match the Fresnel curve for going from low to high IOR. This is done by transforming the angle using Snell's Law.

[peterkutz]

In practice, the key is that, as @fpsunflower described, when going from high IOR to low IOR, you don't evaluate the Fresnel equations directly. You first calculate the refraction angle using Snell's Law and your bending IOR. If the refraction angle is invalid, you have TIR. If the refraction angle is valid, you evaluate the Fresnel equations as if you're hitting the surface from the outside, using the refraction angle and the reflectivity IOR. This ensures reciprocity as any path will have the same reflectivity in either direction by construction, so you can use any function for the reflectivity. In our case, we would want to use the Fresnel equations with an IOR derived from an F0 reflectivity that has been scaled by the specular_weight.

[portsmouth]

The ideas about using the Fresnel reciprocity (Stokes) relation and stretching of the curves are interesting, but for my own edification I found it useful to be a bit more precise about what the full proposal is (including mine). Please forgive the excessive verbosity..

I work through this with my preferred notation in the figure below (as usual, there are several different notation conventions and different people like different conventions, so it can get confusing).

I show the 4 possible scattering configurations that I mentioned.

In detail, I assume:

• There is a well defined exterior/outside "above" (i.e. defined to be towards the outward normal to the surface), and interior/inside "below". (This is really just for the purposes of labelling the IORs -- the physics obviously doesn't care what we consider to be "outside").
• The physical IORs of the interior and exterior are known ($n_i$ and $n_e$ respectively). Their ratio defines $\eta = n_i/n_o$. For now, this corresponds to your "bending IOR" ratio, say. I don't assume here that the interior or exterior has higher IOR (e.g. the exterior could be a higher IOR coat, or ambient medium).
• The reflection Fresnel factor can be written unambiguously (and for all ray configurations) in terms of the cosine of the incident acute angle to the normal and the ratio of the IORs in the transmitted and incident hemisphere, i.e. $F(|\cos\theta_i|, \eta_{ti})$. For entering rays $\eta_{ti} = \eta$, while for exiting rays $\eta_{ti} = 1/\eta$.
• Entering: the incident ray approaches from the hemisphere of the exterior , i.e. $\cos\theta_i > 0$. It reflects into the $R_+$ mode and refracts into the $T_+$ mode.
• Exiting: the incident ray approaches from the hemisphere of the interior, i.e. $\cos\theta_i < 0$. It reflects into the $R_-$ mode and refracts into the $T_-$ mode.
• For shorthand we can refer to the 4 Fresnel reflection and transmission factors as $R_+$, $R_-$, $T_+$, $T_-$.
• Energy conservation demands that $T_+ = 1 - R_+$, and $T_- = 1 - R_-$.
• If the incident ray is in the hemisphere with higher IOR, TIR may occur.
• The Stokes reciprocity you talked about ensures that if there is no TIR, then $F(|\cos\theta_i|, \eta) =F(|\cos\theta_t|, 1/\eta)$. (So the ray that enters along the refraction direction reversed, reflects the same amount as the original reflection..).

Current behaviour

The specular_weight functions currently by modifying $n_i$, thus producing a modified $\eta(\xi_s)$ (such that $\eta(\xi_s)\rightarrow 1$ as specular_weight $\xi_s\rightarrow 0$).

This is computed by requiring that the resulting Fresnel factor of $R_+$ at normal incidence is $\xi_s$ times the original Fresnel factor (clamped if $\xi_s > 1$).

Proposal I: Modify exterior reflection only

Given this setup, my former proposal for the effect of specular_weight $\xi_s$ is simply:

• Replace the Fresnel factor of $R_+$ with $F(|\cos\theta_i|, \eta(\xi_s))$, but leave everything else the same. In other words, only the reflection viewed from above is altered.
• In particular, the ray refracts the same as before (both into and out of the material), with the same transmission coefficients.
• Also the $R_-$ mode is unaffected, so the reflection viewed from below is not reduced or tinted. This seems to accord with the usual use case for this where we don't want specular_weight to alter anything but the observed reflection from outside.
• Energy conservation would require that $\xi_s \le 1$, as the transmission is not altered to compensate.
• The intention is that this works at the microfacet level too, so the microfacet multiple scattering calculation is affected accordingly since the Fresnel factor has changed (as it would be strange for example if the single scattering reflection is completely killed off, but a multiple scattering lobe still remains).
• Note this is quite similar to how we say specular_color tinting currently functions (in that case, it multiplies $R_+$ and leaves everything else the same). We didn't mention whether this is applied at every bounce during microfacet multiple scattering, and really should clarify that. If it does, there will be extra saturation of the color for rough materials. Alternatively we could say that the tint is to be multiplied into the macroscopic BRDF (that is, the $R_+$ mode of it only), so there would be no saturation.

Proposal II: Decoupled IORs

I'll attempt to understand at a similar level of detail what this proposal is, to check if that matches your ideas.

The only thing you need to mess with is the implementation of the F term. Instead of taking in cos_theta and eta as arguments, you should provide cos_theta, bending_ior and specular_ior.

I assume "specular IOR" is the modified ratio $\eta(\xi_s)$, and "bending IOR" is the original unmodified IOR i.e. $\eta(1)$.

When entering a material, you just evaluate the fresnel math directly with cos_theta and specular_ior, there's no TIR to worry about.

So for you, "entering" means transmitting from the lower to the higher IOR. I don't define entering that way since $n_i < n_e$ is allowed due to a coat or nesting (though more typically $n_i > n_e$). But we just have different language here I think.

The point is that TIR is possible if the ray is incident from the higher IOR hemisphere ("exiting" for you). Let's assume this is the more typical case where $n_i > n_e$ (i.e. $\eta = n_i/n_e > 1$). So then entering/exiting rays mean the same for you as for me.

So for the entering case, the $R_+$ mode cannot exhibit TIR. So then we just evaluate $F(|\mu_i|,\eta(\xi_s))$, giving the modified reflection coefficient.

When exiting a material, you first evaluate the "squeezed" refraction angle (and TIR) with the bending_ior and as long as there is no TIR, you evaluate your fresnel curve using the squeezed angle and your specular_ior value. The identity I showed above is helpful because it shows that this is equivalent to evaluating the fresnel math "directly" from the raw equations. In my description above, when I used bending_ior and specular_ior I should have made it more clear these should be the ratio between internal and external IORs. When the ratio is >1 you are "entering" and when <1 you are "exiting" regardless of what the geometry looks like. Like I mentioned above, the most important thing to realize is that the Fresnel equation when exiting is the same as entering just squeezed (with a clamp to 1.0 as soon as you are past Brewster's angle).

This is the $R_-$ mode, which since $\eta = n_i/n_e > 1$ can undergo TIR. TIR occurs for incident angles such that $\cos^2\theta_i \le 1 - 1/\eta^2$.

If there is TIR, $R_- = 1$.

If there is no TIR, then we can apply the reciprocity relation to obtain $F(|\cos\theta_i|, 1/\eta) = F(|\cos\theta_t|, \eta)$ (for some valid $\cos \theta_t$ in the upper hemisphere).

(I'll note that this relation only works if the interface is smooth of course, so this is presumably happening at the microfacet level).

The unphysical modification is then to replace the reciprocal Fresnel term with $F(|\cos\theta_t|, \eta(\xi_s))$. Or more generally the function $F(\mu, \eta)$ can be replaced with something like an arbitrary Schlick curve defined only for $\mu\ge 0$ and $\eta\ge1$.

Some questions arise:

• Are the transmission coefficients $T_+$ and $T_-$ affected? Perhaps the idea is (I hope) that the energy conservation relations $T_+ = 1 - R_+$, and $T_- = 1 - R_-$ still hold after the modification to $R_+$ and $R_-$. In which case, changing the specular weight will alter the energy of the refracted beam to compensate for the change in reflected energy. Or perhaps instead only the reflection coefficients are supposed to change, so total energy is lost for $\xi_s < 1$ or gained for $\xi_s > 1$? (If the latter interpretation is used, I think this means we probably should forbid the $\xi_s > 1$ case).

• Do we actually want the reflection $R_-$ (i.e. seen from the interior/below) to be modified? Perhaps it makes sense that the reflection from below is also killed as the weight goes to zero. Though in this proposal the TIR is preserved. So then for zero weight, one will still see the effect of TIR from the interior surface of a piece of solid glass. In the current behaviour of the spec, of course zero weight produces IOR 1, so there is zero reflection from above or below. (Probably this if fine though I suspect, as the TIR modes of the interior will be interpreted, seen from the outside, merely as an alteration of the refracted image, not as a "highlight").

• How should we think about the microfacet multiple scattering? It doesn't seem to make sense to just not account for the weight or color in the multiple scattering lobe. If this applies at every interaction with a microfacet, then the multiple scattering calculation will have to be changed accordingly (as the Fresnel factors are involved in a non-linear way it doesn't amount to a simple factor). As noted, this is not actually totally clear even in the current statement of the spec due to the specular_color, and we need to clarify it. A tentative suggestion is to stipulate that all of this Fresnel modification due to the weight is explicitly supposed to be taken into account in the multiple scattering approximation, so then the multiple scattering lobe will naturally die off as the weight goes to zero. But for the color, we just say that it functions simply as a scale factor of the macroscopic BSDF (a question though is should only the reflection from above be tinted, or the reflection from below as well? Currently we say the former, though we're not clear about the macro versus micro aspect).

• The use case you mentioned of having a low bending IOR and high specular IOR (to effectively have glossy reflection but specular refraction), could only be achieved via specular_weight $\xi_s > 1$, in the current parametrization. So this would require the user to dial in a low specular_ior for the refraction, then artifically boost the specular_weight to effectively get the rough reflection. Maybe that's a reasonable workflow, though it seems a bit unintuitive.

• We have to worry about the fact that the coat or ambient medium may have higher IOR than the base. In that case the TIR occurs for rays incident from the exterior (in my definition). In that case the same process applies using the reciprocity with the reflection from the interior.

You are right that we should spell out exactly what the interaction with the coating should be. As long as you take the proper ratio for both quantities, you can preserve the physical behavior. You also gain additional flexibility if you want it, but we don't need to reinvent the wheel. Whatever language we already have in the spec could just be extended to cover both IOR ratios.

Another area that will need to be updated is the section on dispersion. I have to admit I haven't implemented this part in a decoupled setting myself, but I suspect that again we could just view the IOR tweak that dispersion implies as a tweak to both the specular IOR and the bending IOR.

• To avoid confusion in a practical implementation I would probably want to have the function which computes the Fresnel factor (and the dispersion calculations) be aware of the specular_weight, rather than supply two IOR ratios. So essentially query the local surface for its specular weight, and hand that off to the Fresnel evaluation together with the physical IOR ratio. Then (it seems) everything else can be determined inside the Fresnel function or dispersion logic.

[portsmouth]

Another proposal is (with a sub-optimal name perhaps, but just to distinguish from "decoupled IORs").

Proposal III: Decoupled R and T

• Compute both reflection coefficients $R_+$ and $R_-$ using the modified IOR ratio $\eta(\xi_s)$.
• Compute both transmission coefficients $T_+$ and $T_-$ (and the refraction directions) using the original IOR ratio $\eta(1)$.

This seems like a more direct implementation of the decoupling between the reflection and refraction.

The advantage of this is that all the reflection (interior and exterior) goes completely to zero as the weight goes to zero, and there is no special case logic needed for the TIR mode. The TIR modes naturally go away (pushed to extreme grazing) as the specular weight $\xi_s \rightarrow 0$. In contrast in the "decoupled IORs" approach, at $\xi_s \rightarrow 0$ the TIR remains, which seems like it would possibly be considered an artifact (a bright interior reflection, despite no exterior reflection).

The refraction direction and energy, seen from both sides, is explicitly untouched (as noted, this is not clear in the current Proposal II). In order to avoid energy gain, this does require that $\xi_s \le 1$ (though the "decoupled IORs" approach also has this requirement I think).

Also the issues about how to calculate multiple scattering are similar to the other proposals.

In this proposal though the TIR experienced by the R and T modes doesn't align though. Possibly this leads to a visual artifact, as for example as the R modes go to zero, there are areas where the T modes also go to zero due to TIR, thus the ray is killed. The Proposal II was designed to ensure that energy can in principle be preserved.

(Just to note, I think most likely the Proposal II makes more sense, but perhaps it was worth mentioning this possibility for discussion).

[portsmouth]

The standard BSDF for a dielectric interface is described in "Microfacet Models for Refraction through Rough Surfaces" by Walter et. al 2007.

This gives the reflection micro-BRDF as:

And the refraction micro-BTDF as:

They also note:

We ought to consider how the proposal will alter these standard formulas.

I take it the Fresnel factor in these formulas is replaced as described above. Perhaps this maintains energy conservation (and that reciprocity relation) after the modification? We should check.

[fpsunflower]

Yup, only the Fresnel term $F$ is altered in the microfacet formula. The impact on the overall energy conservation seems small in my experience, so I just re-used the albedo tables baked with an $F_0$ that matches $\eta$ and re-use them even when $F_0$ is something else.

[portsmouth]

Yup, only the Fresnel term F is altered in the microfacet formula. The impact on the overall energy conservation seems small in my experience, so I just re-used the albedo tables baked with an F 0 that matches η and re-use them even when F 0 is something else.

OK sure, so I interpret this as meaning that:

• the Fresnel term is modified in the macroscopic dielectric BRDF and BTDF.
• the Fresnel in the BRDF is changed according to the "reciprocity" relation, i.e. the logic in (*) below.
• This applies to the BRDFs for entering and exiting rays (to/from the base dielectric into the adjacent medium), whether a coat is present between the base and ambient medium, or not.
• The Fresnel in the BTDF is given by 1 minus the Fresnel in the BRDF, maintaining energy preservation in the limit of a smooth glass interface (to be proven, but it seems plausible that this reciprocity based remapping keeps the energy preservation intact).
• For a rough glass interface, the single scattering mode loses energy. The compensation term is not modified to account for the changes to the single scattering Fresnel. Thus it does not preserve energy, but you claim the effect is negligible.

I worry about how negligible the inconsistent multiple scattering compensation really is, in a very rough case. Having this lack of energy preservation (for glass) either explicit in the spec, or present but not quantified, seems unpleasant to me.

I'd prefer an approach where we instead modify the Fresnel terms for the BRDF/BTDF of individual microfacets, using the reciprocity formula. Then I think probably if you follow through the math, this produces the same single single scattering lobe in the macroscopic BRDF/BTDF, but in principle changes the multiple scattering compensation to perfectly preserve energy. That at least would be a fully unambiguous modification, which while using the unphysical Fresnel factors still perfectly preserves energy.

---

(*) If there is no TIR, then we can apply the reciprocity relation to obtain $F(|\cos\theta_i|, 1/\eta) = F(|\cos\theta_t|, \eta)$ (for some valid $\cos \theta_t$ in the upper hemisphere). The unphysical modification is then to replace the reciprocal Fresnel term with $F(|\cos\theta_t|, \eta(\xi_s))$. Or more generally the function $F(\mu, \eta)$ can be replaced with something like an arbitrary Schlick curve defined only for $\mu\ge 0$ and $\eta\ge1$.

[portsmouth]

On second thoughts, um, maybe it is totally fine that energy is not preserved, since the user is asking for that essentially..

Also given that, perhaps it is fine to simply state that this (reciprocity-based) modulation is only applied to the single scattering BRDF term (in the macroscopic BRDFs of entering/exiting rays). The BTDFs are unchanged. The multiple scattering compensation is unchanged. That is then clear and easy to implement.

If specular_weight $\le 1$, the effect of the modulation can then only be to overall lose energy (I think), so it is physically reasonable. (If specular_weight $> 1$, energy is generally not conserved, but this seems a reasonable caveat which we can communicate to users).

One might worry that in the rough limit, the multiple scattering will possibly dominate, so the specular_weight will appear to do very little. But perhaps this is also artistically reasonable, as in some ways the multiple scattering lobe involves both reflection and transmission (and having the reflection multi-scatter darken but not the transmission multi-scatter is probably not a desirable effect).

It's still unclear to me whether it is best to have the Fresnel of both entering and exiting rays altered. Altering both seems more sensible. If the camera is under an ocean viewing the refracted sky through the waves, do we want specular_weight to modulate the internal reflections of the water under-surface, or should it only change the sky reflections seen from above the waves? It seems a bit arbitrary to treat the reflections seen from outside differently.

Also if we don't alter both, then as specular_weight $\rightarrow 0$, the external reflections are killed but not the internal, which might look odd. (Though note that with the reciprocity modulation, TIR is untouched always, even as specular_weight $\rightarrow 0$, so perhaps this is not too bad).

[fpsunflower]

I should add that my experience with energy conservation of decoupled Fresnel was based on Turquin's approach where you just rescale the total albedo (of BRDF+BTDF) to 1.0. In that model the impact of how $F$ splits between reflection/refraction is quite small on the total overall albedo. The only source of energy loss is the masking terms, and those are only impacted by the ray bending IOR (which is what you should use to index the tables). Its not perfect since the balance of reflection and refraction does impact how the masking term is evaluated but in practice its fairly small.

I haven't attempted to implement this in the Imageworks style energy conservation approach with an extra diffuse lobe for energy compensation. Honestly, with hindsight I feel like that approach is overkill. It doesn't match the ground truth any better than Turquin's simple approach and is way more expensive and confusing to implement.

Like you said, if you do real multiple scattering (ala Heitz. et al) between your microfacets, you can have a fully correct result because you are only tweaking the balance on individual facets and the random walk has a weight of 1.0 by design (all the masking stuff cancels out with enough bounces).

[portsmouth]

Just to be clear Chris, in your proposal did you intend that the BTDFs are unchanged, only the BRDFs change?

Also do you change both BRDFs (for entering and exiting), or only for entering?

About the multiple scattering, we could either say the Fresnel alteration happens at the microfacet level, so then in principle the (full, Heitz-style) multi-scatter BRDF will go to zero as specular weight goes to zero (requiring some alteration to the multi-scatter formulas). Or, as in my last comment, we could simply say that the Fresnel alteration happens to the final macro-BRDFs in the single-scatter term only, and the multi-scatter is defined to be the same (full, Heitz-style) result as one gets without any Fresnel alteration (this then requires no alteration to the multi-scatter implementation [*]). Which would you prefer?

[*] Except the Turquin approach would need to use the albedo of the unmodified BRDF, which might be a bit of a gotcha.

[fpsunflower]

Both BRDF and BTDF are changed. Its the same $F$ that shows up in both. If less energy is reflected (as specular tends to 0), the remaining energy gets transmitted.

I think it makes more sense to refer to the individual microfacets for the spec. Multi-scattering would do the "right thing" and single scatter is just the single-bounce specialization of that.

[portsmouth]

Both BRDF and BTDF are changed. Its the same F that shows up in both. If less energy is reflected (as specular tends to 0), the remaining energy gets transmitted.

I see, though that does mean that the specular_weight will impact the brightness of transmission as well as reflection, but presumably it is visually reasonable. What would the issue be if we instead simply stipulated that the BTDFs are unchanged?

I think it makes more sense to refer to the individual microfacets for the spec. Multi-scattering would do the "right thing" and single scatter is just the single-bounce specialization of that.

That sounds like a good approach, so the entire BRDF automatically goes to zero as specular_weight $\rightarrow 0$. It will mean that technically the multiple scattering lobe (in the "true" physical light transport between microfacets, just with modified Fresnel factors) changes depending on the specular_weight. In the Turquin approach, perhaps (like you said) this doesn't make much difference. If doing a diffuse-lobe (Kulla & Conty style) correction then presumably the lobe needs to be modified accordingly, but perhaps it is not too hard to come up formulas if needed.

Also do you change both BRDFs (for entering and exiting), or only for entering?

What about this? Would you have the specular_weight alter the reflection of an ocean surface seen from below, or only from above? (Or less exotically, when computing the BRDF for an internal reflection during pathtracing glass).

Or have only the external reflections modified (if so, why?).

[fpsunflower]

Also do you change both BRDFs (for entering and exiting), or only for entering?

What about this? Would you have the specular_weight alter the reflection of an ocean surface seen from below, or only from above? (Or less exotically, when computing the BRDF for an internal reflection during pathtracing glass).

Or have only the external reflections modified (if so, why?).

I would change it for both. Its simpler and in glass some of what reads as "specular highlights" are actually multiply reflected highlights. For example if your goal was to get rid of specular (specular_weight=0) then you would want all those internal reflections gone as well.

[portsmouth]

I implemented the decoupling scheme in Arnold, with the following results (on a smooth glass shaderball with specular_ior=1.5 and a little dispersion, with varying specular_weight). As expected, it has the desired effect of changing the reflection strength without disturbing the refraction appearance. (And now one can produce that unphysical glass ball with refraction but no reflection - which is a bit weird, but I guess it is useful artistically to have this independent dial for the reflection strength).

(Just to note, the left column is not actually how the currently shipped Arnold OpenPBR functions, since we simply had the transmission unaffected by specular_weight . This was technically in violation of the spec though).

specular_weight	current spec	new
1
0.5
0.0

I committed a draft of the required language in the spec. It's a bit wordy, but I tried to be as clear as possible leaving nothing ambiguous (so e.g. it describes the logic for both entering and exiting cases, since TIR can occur for either due to nested dielectrics or the coat).

@fpsunflower and @peterkutz it would be very helpful if you can proofread this, and suggest any improvements.

--------------------------------------

Some initial commentary:

• I wouldn't say it was entirely trivial to implement.. (It did boil down though to a modified dielectric Fresnel routine, with extra logic for the specular_weight $\ne 1$ case). Some example code might be helpful as a guide (for example in the OpenPBR viewer). Implementing this in the MaterialX graph seems likely to be quite unpleasant.

• Some plots of the Fresnel factors after the specular_weight modulation might be helpful, though could also just make it more confusing and wordy.

• Renders similar to the ones shown above would be good to put in the spec for the intuition.

• The approach in the currently shipped Arnold avoids the refraction changing with specular_weight simply by accounting for the specular_weight IOR modulation in the BRDF only, and ignoring it in the BTDF which uses the unmodified IOR for refraction direction and Fresnel (producing a look not described in the current spec though). It would be good to understand why this more naive approach is not a workable scheme compared to the more "principled" Stokes reciprocity-based scheme (presumably it leads to some violation of energy conservation/preservation due to the mismatch between the Fresnel of reflection and transmission, which manifests as some visual artifact?). Understanding this more clearly might improve the spec, as currently the proposed text lacks a very compelling intuition for why these particular formulas were chosen.

• I admit I also don't fully understand the energy balance after the proposed modulation using the Stokes reciprocity, since we are unphysically altering the Fresnel curves. Normally the refraction squeezes the transmitted light into a cone, and the radiance is boosted accordingly in the BTDF formula to account for this (as in these formulas and reciprocity relation). Does the altered Fresnel still satisfy that reciprocity relation (and is energy overall preserved?), it's not obvious at least. (It's not necessarily bad if not, since specular_weight is unphysical if changed from the default, but would be good to understand it in detail at least). Some numerical calculations might help to check.

• We didn't state it clearly before, but now I made it clear that specular_color functions as a (non-physical) overall tint multiplier of the total macroscopic BRDF (for both entering and exiting rays), leaving the BTDF untouched, thus removing energy. You could also choose for the tint to only apply to entering rays, but it seems unnecessary to me.

• For the interaction with the coat, I kept the external medium IOR generic in the math (i.e. $n_e$). In my implementation, I set this equal to either the coat or the ambient medium IOR (from nested dielectrics). For a partially present coat, the interior/exterior IOR ratio is set to the statistical blend of the coated and uncoated ratios. This is an approximation (since really one should statistically select between the separate physical situations, or blend the BSDFs) but seems a reasonable one.

• The interaction with the thin film is a bit more tricky. I didn't attempt to account for this, but in principle the film sits between the base and the coat, so it should affect the IOR ratios (so specular_weight should in principle change the dispersion fringes as well). It seems an OK approximation to just ignore this though, and treat the dispersion as if the original specular_ior was unchanged, so the fringes are insensitive to specular_weight (as done in my test renders above). We could stipulate that in the spec perhaps, though it seems less physically plausible that way since in reality the dispersion should be correlated with the dielectric reflectivity. We should certainly clarify in the spec how the specular_weight modulation interacts with the dispersion, anyway.

[portsmouth]

I do see a difference between the Arnold approach and the reciprocity-based approach. (Here using the "physical TIR" mode in Arnold, which does a more physically correct Fresnel calculation than the default). The difference is not large, but the reciprocity approach does prevent some dark areas from appearing.

specular_weight	Arnold	new
0.5

[portsmouth]

I admit I also don't fully understand the energy balance after the proposed modulation using the Stokes reciprocity, since we are unphysically altering the Fresnel curves. Normally the refraction squeezes the transmitted light into a cone, and the radiance is boosted accordingly in the BTDF formula to account for this (as in these https://github.com/AcademySoftwareFoundation/OpenPBR/pull/247#issuecomment-2675340452). Does the altered Fresnel still satisfy that reciprocity relation (and is energy overall preserved?), it's not obvious at least. (It's not necessarily bad if not, since specular_weight is unphysical if changed from the default, but would be good to understand it in detail at least). Some numerical calculations might help to check.

I think I better understand the energy balance now. The formulas quoted above are misleading since the delta function is written there in "half-direction" measure, not solid angle measure.

The solid angle measure versions are as quoted in PBRT (this is for a smooth interface, and/or at the microfacet level):

Integrating these over the hemispheres gives the reflection and transmission albedos:

These do not sum to 1, since the radiance of the transmitted rays is boosted by the factor $n^2_t / n^2_i$ (since the energy incident from the upper hemisphere is squeezed into a cone in the lower hemisphere due to the refraction).

In order to satisfy $R + T = 1$ (and restore reciprocity), the light transport has to be done in terms of the "basic radiance" $L / n^2$, where $n$ is the local IOR. Then the albedos for reflection/transmission of basic radiance (say $\tilde{R}$, $\tilde{T}$) will be the quantities that satisfy

Renderers would/should take this into account. (Our discussion of energy conservation in the spec should be altered to state this more correctly).

In any case, the modification we do via specular_weight does not change the geometry of the refraction, it only changes the Fresnel factor $F_r(\omega_r)$, in both the reflection and transmission. This then continues to obey energy conservation (and reciprocity) in the same fashion as the physically correct formulas.

The current Arnold approach in contrast is more incoherent, since altering the BRDF without altering the BTDF does not correspond to merely changing the Fresnel factor, and leads presumably to energy conservation/reciprocity violation.

[peterkutz]

Nice analysis and investigation @portsmouth !

Based on all of the considerations, "Proposal II: Decoupled IORs" does seem like the best approach.

It makes sense intuitively that this approach conserves energy. At each interaction either reflection or transmission occurs with complementary probabilities, so the path is always continued without modifying the throughput. And the scaling of the radiance is due to the density of light energy changing, not the actual amount of energy changing, and it only depends on the degree of refractive squeezing, not the proportion of light transmitted.

It also makes sense that this approach would reduce darkening artifacts compared to the other approaches. Even for other approaches that do not modify TIR, they still modify non-total internal reflection, which can be arbitrarily close to 100% so which can still be quite visible. Avoiding almost-black artifacts seems ideal.

I still need to more carefully consider some of the details you've written about, and I still need to review your updated wording in the spec itself, but at a high level everything sounds good.

I haven't fully caught your proposal for the effect of the exterior IOR on this system yet, but I would assume that the specular weight is applied directly to the relative IOR of the exterior IOR and the original base specular IOR. In the Eclair implementation, we have a reflection/transmission-coefficient abstraction that is treated like a black box by the microfacet lobes, so it seems like it should be possible to modify the reflection/transmission coefficients returned by that abstraction without affecting anything else. Of course the microfacet multiple scattering compensation might need to be adjusted too, but I'll be curious to first see how this looks if leaving that unchanged.

[portsmouth]

Maybe it's worth giving some very simplified C "pseudo" code (compiling, but out of context) for the dielectric Fresnel modification.

Something like, given:

// Standard dielectric Fresnel function, depending on:
//   - mu_i = magnitude of angle cosine of incident ray (mu_i >= 0)
//   - eta_ti = (IOR in hemisphere of transmitted light photons) / (IOR in hemisphere of incident light photons)
float DielectricFresnel(float mu_i, float eta_ti);

The function to compute the modulated Fresnel due to specular_weight is:

//  specular_weight (>= 0) modulates the Fresnel F0 linearly, without disturbing TIR/refraction
float DielectricFresnel(float mu_i, float eta_ti, float specular_weight=1.f)
{
   if (specular_weight == 1.f)
      return DielectricFresnel(mu_i, eta_ti);

   // Compute modified IOR ratio
   float F0 = sqr((eta_ti - 1.f)/(eta_ti + 1.f));
   float eps = copysignf(min(1.f, sqrtf(clamp(specular_weight * F0, 0, 1)), eta_ti - 1.f);
   float eta_ti_prime = (1.f + eps) / max(FLT_EPSILON, 1.f - eps);

   if (eta_ti_prime >= 1.f) // (No TIR possible)
      return DielectricFresnel(mu_i, eta_ti_prime);

   // In the possible-TIR case, check for TIR and if not, use the "un-squeezed" Fresnel curve.
   float mu2_t = 1.f - (1.f - sqr(mu_i))/sqr(eta_ti);
   if (mu2_t <= 0.f)
      return 1.f; // (TIR occurs)
   return DielectricFresnel(sqrtf(mu2_t), 1.f/eta_ti_prime);
}

That's easier to understand than the formulas perhaps. (Maybe there is a simpler/nicer way of implementing it than this, but this is the most straightforward way it seems).

It would be easy to add a folder with some such example code snippets in it for reference from the spec.
Similar C pseudo-code snippets for e.g. the coat darkening, and roughening math, are probably helpful to supply.

[portsmouth]

I haven't fully caught your proposal for the effect of the exterior IOR on this system yet, but I would assume that the specular weight is applied directly to the relative IOR of the exterior IOR and the original base specular IOR.

I expressed the formulas in terms of a generic exterior IOR $n_e$. The dielectric IOR (i.e. IOR of the interior) is $n_i$ = specular_ior. Their ratio is $\eta = n_i/n_e$.

The dielectric exterior IOR $n_e$ is either the "ambient" medium IOR $n_a$ (which could be arbitrary depending on dielectric nesting), or the coat IOR $n_c$ = coat_ior. The surface has a statistical mixture of these two situations according to the coat_weight.

To deal with this, the spec suggests to use an effective $\eta$ according to the coat weight:

\eta = \mathrm{lerp}(n_i/n_a, n_i/n_c, \mathtt{C})

where $\mathtt{C}$ is coat_weight.

That's not really correct (instead one should ideally e.g. stochastically decide whether coat is present or not, not use a blend of IORs), but it seems a reasonable practical approach.

[portsmouth]

For the MaterialX implementation of this, it doesn't seem possible without modifying the MaterialX spec, since the Fresnel function has to change. We can propose something for that.

But in the interim, a possible approximation we can easily do in the MaterialX graph is to just ignore the IOR modulation in the dielectric BTDF. (Which is what the current Arnold implementation does, as noted). This will violate the proposed decoupling behavior in this PR, but is not very different in appearance (as shown above).

I'd suggest we go with that in order to get this merged, and revisit the MaterialX side when possible.