Anti Aliasing is not working as intended
See discussion #20290
User reports:
I'm now looking at the behavior for the official, standalone VisIt application. On Mac I don't get anti-aliasing whether the checkbox is checked or not. On windows I'm always getting antialiasing. Based on our unit tests, I believe Linux is also always giving antialiasing.
To change the setting I am toggling the check box, saving settings, and restarting VisIt.
@biagas says that antialiasing toggle logic is not working as intended, so choices the user makes to the setting in the rendering options have no effect. It sounds instead like depending on what kind of system you have, AA may be turned on or off by default.
See linked discussion for more information and potential solutions.
Here's the core issue that I'm running into on my 2019 iMac with Intel Core i5, Radeon Pro 560X.
Running the official build of stand-alone VisIt, I see anti-aliasing on screen and if I do a "save window" with the "screen capture" option turned on. If I turn off the "screen capture" option, then the resulting image does not have anti-aliasing.
As far as I can tell, the proper anti-aliasing flags are being set all the way down the chain to where the actual render call gets made into the underlying vtk code, but it's quite possible I'm missing something.
It appears that there are some hardware differences between different Mac models that can affect anti-aliasing behavior. We haven't seen problems on the newer M-chip macs, but an older mac mini doesn't do anti-aliasing on screen or in saved images. The iMac I mention above is a mixed bag.
We will investigate when we get a chance and get to the bottom of the myriad anti-aliasing issues.
We will look into AA options with VTK 9.4 / 9.5 and see if things are plumbed correctly. Not sure how we can track down all the various issues with different platforms, we will have to experiment.
FWIW, I found this comment in QVTKOpenGLNativeRenderWindow, which I believe is being used as the underlying render object by vtkQtRenderWindow:
// When a QOpenGLWidget is told to use a QSurfaceFormat with samples > 0,
// QOpenGLWidget doesn't actually create a context with multi-samples and
// internally changes the QSurfaceFormat to be samples=0. Thus, we can't
// rely on the QSurfaceFormat to indicate to us if multisampling is being
// used. We should use glGetRenderbufferParameteriv(..) to get
// GL_RENDERBUFFER_SAMPLES to determine the samples used. This is done by
// in recreateFBO().
Naturally, there is no other mention of GL_RENDERBUFFER_SAMPLES or recreateFBO() in vtk 9.2.6, but that method does exist in vtk 8.1.0.
Antialiasing via MSAA is ON by default for Linux and Windows, off for Mac. vtkOpenGLRenderWindow has a static method, SetGlobalMaximumNumberOfSamples, that is called with 0 for Mac, 8 otherwise.
Can possibly turn MSAA on for Mac by calling this method with a number higher than 1 before Context creation. I don't have a Mac to test with, so don't know if it will work.
I have a branch that modifies VisWinRendering::SetAntialiasing to call vtkRenderWindow::SetMultiSamples with 0 to turn it off , 4 or 8 to enable. This basically reactivates our antialiasing toggle in the Rendering options window.
The changes work just fine for on-screen windows, but if the render window is a vtkOSOpenGLRenderWindow (as is the case when we run our regressions), then nothing is renderered. This is bad.
databases/CGNS.py test turns on antialiasing and exhibits the problem when run from my branch.
Currently investigating if it is possible to get antialiasing in vtkOSOpenGLRenderWindow by another means.
We found that setting SetGlobalMaximumNumberOfSamples() did enable AA on most macs, although we had some older iMacs that did not respond to it.
I can help with Mac testing if you'd like.
I can help with Mac testing if you'd like.
That would be great! I have a few changes to make to my branch, will let you know when it is ready for testing.
@marcdurant the branch is: biagas/fix_antialising_flag_response
When antialiasing is turned on it will use FXAA if render window is a vtkOSOpenGLRenderWindow, MSAA otherwise. I am working on making the antialiasing option be multi-modal so that you can choose for yourself between FXAA and MSAA.
Let me know if any of this helps on Mac, and also if anything I changed causes problems on Mac.
@marcdurant FYI, the branch is based off develop, which now requires VTK 9.5.0. A few other Third party library versions have also been updated (conduit, mfem,openexr,vtk-m)