Unable to use NEAREST scaling in filter (broke some time after 0.10.1)

[LummoxJR]

In my application I was using version 0.10.1 until fairly recently. In order to get high-quality upscaling of pixel art, I setup my stage like so:

stage
- hqSprite (with FlattenFilter)
-- mainSprite (with PixelFilter)
--- Scene objects

And PixelFilter was defined like so:

class PixelFilter extends BitmapFilter {
    BitmapFilter clone() => new PixelFilter();

    void renderFilter(RenderState renderState, RenderTextureQuad renderTextureQuad, int pass) {
        renderTextureQuad.renderTexture.filtering = RenderTextureFiltering.NEAREST;
        // was previously renderState.renderQuad, but the name changed
        renderState.renderTextureQuad(renderTextureQuad);
    }
}

The hqSprite bit may have been overkill here. Anyway, in my application whenever the stage is resized, I calculate a new zoom level for mainSprite.

if(hq) {
    int clientWidth = _canvas.clientWidth;
    int clientHeight = _canvas.clientHeight;
    int w = math.max(fullWidth,1), h = math.max(fullHeight,1);
    int zoom = math.max(1, math.min(clientWidth/w, clientHeight/h)).round();
    sourceWidth = (fullWidth * zoom).round();
    sourceHeight = (fullHeight * zoom).round();
    mainSprite.scaleX = mainSprite.scaleY = zoom;
}
else {
    sourceWidth = fullWidth;
    sourceHeight = fullHeight;
}

This worked like a charm in 0.10.1; the mainSprite would get scaled up using NEAREST filtering. But now in 0.13.2, this no longer works as expected. The NEAREST filter mode is being ignored.

I'm not sure this is a bug per se, just that the rendering behavior has changed. The problem for me is, I don't know how I can apply the NEAREST filter otherwise in a way that will impact the scene as a whole. Any advice on how to update my code to achieve that effect would be much appreciated--as would any info on how the solution may impact performance.

[bp74]

Sorry for this breaking change! But this is a very interesting use case.

I think the problem is caused by this line: https://github.com/bp74/StageXL/blob/master/lib/src/engine/render_context_webgl.dart#L209

var pixelRatio = math.sqrt(renderState.globalMatrix.det.abs());

Previously the temporary render textures for filters had the same size as the display object without it's global scale applied. The problem with this is that filters look bad on HiDPI displays like phones or Apple Notebooks. Therefore filters are rendered to the temporary textures with a size that matches the global scaling. You could try to replace this line with something like this:

var pixelRatio = 1.0

If this solves your problem we could think about a generic way to configure this behavior. I have to think a little bit more about this before i can give you a final solution. Hope this helps!

[bp74]

Btw. i don't think you have to control the scaling of your main Sprite on your own. You can take a look at the different Stage scale and align settings which should do this automatically.

http://www.stagexl.org/docs/wiki-articles.html?article=stagescale

Or you can take a look at one of the samples, like this one:

https://github.com/bp74/StageXL_Samples/blob/master/example/extension/spine/index.dart

var canvas = html.querySelector('#stage');
var stage = new Stage(canvas, width:480, height: 600);
var renderLoop = new RenderLoop();
renderLoop.addStage(stage);

Here the Stage get's a logical size of 480x600. The canvas element is set up in a way to always fill the whole window, but the coordinate system of the Stage will always be 480x600

[LummoxJR]

Yep, the pixelRatio setting fixed it. (And honestly I'm not surprised this broke, because my filter isn't so much acting like a true filter; my goal was simply to find a way to use a temporary texture.) I'm not averse to making modifications on my own, although I'd like to make sure I have finer control over this one. Ideally I want to use the default behavior everywhere else, and only use this for the one sprite I want to scale with NEAREST. So I'm interested in whatever change you come up with, and I can always apply that manually until upgrading again at a later time.

I'm actually using the stage scaling also. (This is for the BYOND webclient.) We use SHOW_ALL by default, but also can use NO_SCALE and NO_BORDER. It'd be nice to specify an exact zoom level, but this pre-scaling system I have could, now that I think of it, probably do that for me--at least in integer zoom cases, which are the most common.

The reason I use the NEAREST scaling on the scene is that with many users' games, we found--in our software and in the webclient both--regular upscaling produced horrible blurring on games that were meant to preserve pixel art detail. Some games even use a high zoom level just to produce a pixellated effect. The solution I found was to render the whole scene to a texture, stretch that texture by an integer zoom factor using the NEAREST scaling, and then let the result get scaled up or down to the final size with LINEAR. This method not only makes upscaling look better, but it eliminates a lot of ugly artifacting at just-below-1:1 scaling. If I walk around a 2D world with floor tiles that have light and dark edges, the artifacting is extreme without this intermediate step.

[LummoxJR]

I have a proposed solution that I'm currenly implementing as a stopgap.

In the RenderObject class (render_object.dart):

  num get pixelRatio;

In _RenderTextureQuadObject (same file):

  final num pixelRatio = null;

In DisplayObject (display_object.dart):

  num _pixelRatio;
  num get pixelRatio => _pixelRatio;
  set pixelRatio(num r) {_pixelRatio = r;}

And finally, in renderObjectFiltered() in render_context_webgl.dart:

    var pixelRatio = renderObject.pixelRatio is num ? renderObject.pixelRatio : math.sqrt(renderState.globalMatrix.det.abs());
    if(pixelRatio.isNaN) pixelRatio = 1;

This allows me to set pixelRatio=1 for mainSprite. For all others it's null, and the default behavior is respected. (I found out BTW that I was able to lose hqSprite altogether. The stage still needs FlattenFilter for this to look right below 1:1 scaling.)

That isNaN line is one I had added earlier, because I found that without it my project was crashing after I migrated from 0.10.1.

[bp74]

Thanks for your suggestions and i will think about it over the weekend. The most important thing is that the temporary solution fixes your problem for now.

[LummoxJR]

That it does. Your help is much appreciated.