Antialiasing has no effect when ClearColor is transparent
Description of the bug
I've tried with both (SMAA and FXAA), but any looks correct effect when the renderer ClearColor is transparent.
To Reproduce
renderer.setClearColor(0xffffff, 0)
...
const fxaaEffect = new FXAAEffect()
fxaaEffect.subpixelQuality = 0.1
...
const smaaEffect = new SMAAEffect()
smaaEffect.preset = SMAAPreset.ULTRA
smaaEffect.edgeDetectionMode = EdgeDetectionMode.COLOR
Expected behavior
Antialiasing have the same effect when ClearColor have non-transparent value.
Screenshots
No-transparent ClearColor.

Transparent ClearColor.

NOW, if I use lower values in alpha at clearColor the thing gets better.
renderer.setClearColor(0xffffff, 0.1)

But the background is annoying for the application that I needed.
Library versions used
- Three: "^0.132.2"
- Post Processing: "^6.28.4",
Desktop
- OS: [e.g. Windows]
- Browser [e.g. Chrome X.X.X.X]
- Graphics hardware: [e.g. NVIDIA]
Mobile
- Device: [e.g. iPhone]
- OS: [e.g. Android]
- Browser [e.g. Firefox X.X.X]
Repro: https://codesandbox.io/s/affectionate-johnson-0lu79i?file=/src/App.js The problem is that the edge detection logic in FXAA and SMAA ignores the alpha channel. I'm not sure yet how this should be handled.. maybe multiplying the color samples with the respective alpha values could work. I'll look into it.
Unfortunately, I was unable to get FXAA/SMAA to work with a transparent canvas. Anti-aliasing works by blending pixels with neighboring pixels, but when the canvas is transparent, the AA won't have access to the pixels that are "behind" the canvas. The result is an incomplete AA-blend that doesn't remove the jaggies.
I tried using MSAA with a transparent canvas as well, and found that it has the same problem:
Closing this as I don't think this can be solved. Feel free to reopen/comment if you or someone else knows a solution.