pygfx icon indicating copy to clipboard operation
pygfx copied to clipboard

Improve the interpolation filter in flush

Open almarklein opened this issue 8 months ago • 1 comments

Closes #1118

Motivation

The practical improvements of this work are:

  • A proper way to see individual pixels for low pixel ratios.
    • i.e. without the quirks in #1118
    • Simple nearest-neighbour.
    • Also a filter to show a disk for each pixel.
  • Less smoothing when upsampling/downsampling.
    • We used a Guassian kernel before this PR. Gaussian's are awesome, but not so much for interpolation.
  • Better reconstruction when upscaling.
    • On old hardware one could use a low pixel_ratio, apply fxaa, then upscale, and get reasonably pleasing results.
  • Better reconstruction when downscaling (ssaa).
    • The results when doing ssaa, with ratio's like 1.5 or 1.33 are expected to look better.
  • More performant that the current solution.

Tasks

  • [x] Can now use templating in the effect pass shaders.
  • [x] Refactor the OutputPass to use a cubic Mitchell filter by default.
  • [x] Also supports box (nearest), pyramid (linear), and Gaussian filters.
  • [x] Also add a disk filter to see individual pixels, might be nice for debugging or artistic effects.
  • [x] I repurposed the renderer.pixel_filter to set the texture type (nearest, linear, disk, gaussian, or cubic).
    • Previously this was a float indicating relative filter strength.
    • The values 0 and 1 are still allowed, for backwards compatibility, resolving to nearest and cubic, respectively.

Notes

The texture access in the shader is implemented with a templated loop: I've found that doing it in a wgsl for-loop, the performance is much worse, because wgsl does not yet unroll loops, and can therefore not optimize/combine the multiple texture fetches.

The new FilterType enum is implemented using Literal, an idea from https://github.com/pygfx/wgpu-py/issues/720. Let's see how that works out.

Since the old version applied a tiny bit of blurring, even if the source and target texture were of the same size, and the current version does not, all validation screenshots need to be updated 🤷

Example

The basic line example, with aa off, pixel_ratio set to 0.1.

With renderer.pixel_filter = 'nearest':

image

With renderer.pixel_filter = 'disk':

image

With renderer.pixel_filter = 'gaussian':

image

With renderer.pixel_filter = 'cubic':

image

almarklein avatar Jun 19 '25 07:06 almarklein

Looks like we have some sort of race condition with the screenshots again ... the validate_text_md examples is reported as broken, even though I updated to the latest generated screenshot. The generated screenshot is also different from the screenshot produced by the test-build (the one included in the diff images zipfile). There is just one pixel that has a slightly different value, but its consistent.

almarklein avatar Jun 19 '25 10:06 almarklein

@Korijn ready from my end (finally)

almarklein avatar Jun 26 '25 10:06 almarklein

Seems like it was a bit challenging but an important change!

Yeah, loads of subtleties. Like the odd vs even kernels ... and I spend quite some time on good tests.

almarklein avatar Jun 26 '25 11:06 almarklein

For those interested, here's some interesting background on the filters:

  • https://therealmjp.github.io/posts/msaa-resolve-filters/
  • https://bartwronski.com/2022/03/07/fast-gpu-friendly-antialiasing-downsampling-filter/

almarklein avatar Jun 26 '25 11:06 almarklein