Why opacity compensation on the 3D filter? This makes small Gaussians non-opaque.

[f-dy]

The justification for the 3D filter in the paper comes from the fact that "a primitive smaller than 2Tˆ may result in aliasing artifacts during the splatting process, since its size is below twice the sampling interval."

In practice, this means that there is a minimum Gaussian scale (filter_3D in the code), and the real scales are computed from the scales parameters that way:

        scales = torch.square(scales) + torch.square(self.filter_3D)
        scales = torch.sqrt(scales)

However, the code actually implements it as a 3D Gaussian filter, and also applies opactity compensation in 3D:

        scales_square = torch.square(scales)
        det1 = scales_square.prod(dim=1)
        
        scales_after_square = scales_square + torch.square(self.filter_3D) 
        det2 = scales_after_square.prod(dim=1) 
        coef = torch.sqrt(det1 / det2)
        return opacity * coef[..., None]

This last part (opacity compensation), in my opinion, is wrong, because this prevents small Gaussians from being 100% opaque.

For example, a round Gaussian that would exactly have the "parameter" scale filter_3D, and thus the "real" scale sqrt(2)*filter_3D, and a "parameter" opacity of 1, will have a "real" opacity of...

det1 = (filter_3D**2)**3
det2 = (2*filter_3D**2)**3
opacity = 1 * sqrt(det1/det2) = sqrt(1/8) = 0.35

35% !

The effect is similar on larger Gaussian: no Gaussian can be 100% opaque with this 3D opacity compensation.

Conclusion: The 3D filter should really be only about setting a minimum size, without adjusting the opacity, so that small Gaussians can be opaque.

Of course, I totally agree with opacity compensation on 2D Gaussians (which I proposed in Oct 2023 https://github.com/graphdeco-inria/gaussian-splatting/issues/294#issuecomment-1772688093)