Rendering vertical lines issue while camera movement (pixel bleeding)

[jacobranjit]

trafficstars

Current bug behaviour

Expected behaviour

Vertical lines issue in web whenever camera moves, in android as APK there is no issue. Those vertical lines exist only for the web.

https://user-images.githubusercontent.com/52130837/143884831-18c50e76-03c2-4afe-8829-4437223b542d.mp4

Steps to reproduce

https://beach-website.web.app/#/ use arrow keys to move the player

repo: https://github.com/Jcupzz/beach_hack_website_2022

Flutter doctor output

Output of: flutter doctor -v

More environment information

flame: ^1.0.0-releasecandidate.16 flame_tiled: ^1.0.0-releasecandidate.15

Log information

Enter log information in this code block

More information

https://github.com/flutter/flutter/issues/14288

[phyohtetarkar]

I think it is texture bleeding. May be you can try adding padding to your texture when packing.

[jacobranjit]

I think it is texture bleeding. May be you can try adding padding to your texture when packing.

I tried adding extrusion of 1 pixel and 2 pixel to each tiled sprite using texturepacker...it doesn't work. I think the reason is due to drawAtlas method which doesn't work for flutter web. Flutter doesn't support drawAtlas method for web. The site works perfectly for android but doesn't work for web

[spydon]

@Jcupzz have you tried this with 1.0.0? The fallback was removed in the stable release, but that said, you can get those artifacts with drawAtlas too. See #1153.

[jacobranjit]

@Jcupzz have you tried this with 1.0.0? The fallback was removed in the stable release, but that said, you can get those artifacts with drawAtlas too. See #1153.

@spydon Yes, I have tried with 1.0.0 still the problem persists. Well, in android the same code is working perfectly there are no vertical lines issue. So I think with drawAtlas I don't get those artifacts

[leeflix]

I also discovered this bug some time ago... I wanted to make a tile based game with Flutter / Flame, but can't do it because of this. I opened issues on this in the Flutter (https://github.com/flutter/flutter/issues/74127) and Flame #637 repo some time ago with no success. There is also a big issue (https://github.com/flutter/flutter/issues/14288) on this in the flutter repo which was opened in 2018 but is not resolved yet...

I think it hast to do with the methods canvas.scale and canvas.translate. Every plattform even Android is affected (just scale the canvas by something like 1.32). It just happens that certain screen sizes with the right scaling dont produce this bug. For example when I move Mario in the game of @Jcupzz I have no problems only when i resize the browser windows from full screen. (I have a 4k monitor)

It would be cool, if we find a fix / workaround for this, because a friend of mine an me really want to make a tile based game with Flame.

The following is a minimal example to reproduce this (resize the window if it doesn't happen):

import 'package:flame/game.dart';
import 'package:flutter/material.dart';

void main() => runApp(GameWidget(game: MyGame()));

class MyGame extends FlameGame {
  Paint p = Paint()..color = Colors.blue;

  @override
  void render(Canvas canvas) {
    super.render(canvas);

    int tilesInX = 16;
    int tilesInY = 16;
    double tileSize = 16;

    canvas.scale(size.x / (tilesInX * tileSize));

    for (int x = 0; x < tilesInX; x++) {
      for (int y = 0; y < tilesInY; y++) {
        canvas.drawRect(
          Rect.fromLTWH(
            x * tileSize,
            y * tileSize,
            tileSize,
            tileSize,
          ),
          p,
        );
      }
    }
  }
}

[spydon]

@felithium I don't think there is much we can do unfortunately, since it seems to be a Flutter bug. We can of course try to fix it upstream, but I wouldn't have a clue of where to start even... Or if there is some kind of work around that we can do maybe, where the tiles are overlapping with one pixel for example?

[leeflix]

Hey, I might have found a solution. After spending hours trying different solutions to find a workaround (which I actually did aswell) I found out that this problem does not appear when you exactly use canvas.drawRectImage AND set the isAntialias of the Paint instance to false. If you for example use canvas.drawImage it simply does not matter if isAntialias is true or false, the artifacts will eventually appear. I have no clue why that is...

I fixed the game of @Jcupzz by stepping in the source of Flame where the TiledMap is painted. And it does use canvas.drawRectImage, but isAntialias is not set, so by default true, so I set it to false manually.

This is the file but the latest version is slightly different than the version of @Jcupzz: https://github.com/flame-engine/flame/blob/main/packages/flame/lib/src/sprite_batch.dart

void render(
    Canvas canvas, {
    BlendMode? blendMode,
    Rect? cullRect,
    Paint? paint,
  }) {
    paint ??= Paint();

    if (kIsWeb) {
      for (final batchItem in _batchItems) {
        paint..blendMode = blendMode ?? paint.blendMode;

        canvas
          ..save()
          ..transform(batchItem.matrix.storage)
          ..drawRect(batchItem.destination, batchItem.paint)
          ..drawImageRect(
            atlas,
            batchItem.source,
            batchItem.destination,
            paint..isAntiAlias = false, // i changed this
          )
          ..restore();
      }
    } else {
      canvas.drawAtlas(
        atlas,
        _transforms,
        _sources,
        _colors,
        blendMode ?? defaultBlendMode,
        cullRect,
        paint,
      );
    }
  }

I couldn't produce the artifacts anymore.

https://user-images.githubusercontent.com/41200990/147857916-af79ac0d-3c3f-431b-b18e-6f63601e046f.mp4

Would be cool if someone else could confirm this, because I am not 100% if I am not seeing ghosts after spending to long on this problem.

Hope this helps!

[jacobranjit]

Hey, I might have found a solution. After spending hours trying different solutions to find a workaround (which I actually did aswell) I found out that this problem does not appear when you exactly use canvas.drawRectImage AND set the isAntialias of the Paint instance to false. If you for example use canvas.drawImage it simply does not matter if isAntialias is true or false, the artifacts will eventually appear. I have no clue why that is...

I fixed the game of @Jcupzz by stepping in the source of Flame where the TiledMap is painted. And it does use canvas.drawRectImage, but isAntialias is not set, so by default true, so I set it to false manually.

This is the file but the latest version is slightly different than the version of @Jcupzz: https://github.com/flame-engine/flame/blob/main/packages/flame/lib/src/sprite_batch.dart

void render(
    Canvas canvas, {
    BlendMode? blendMode,
    Rect? cullRect,
    Paint? paint,
  }) {
    paint ??= Paint();

    if (kIsWeb) {
      for (final batchItem in _batchItems) {
        paint..blendMode = blendMode ?? paint.blendMode;

        canvas
          ..save()
          ..transform(batchItem.matrix.storage)
          ..drawRect(batchItem.destination, batchItem.paint)
          ..drawImageRect(
            atlas,
            batchItem.source,
            batchItem.destination,
            paint..isAntiAlias = false, // i changed this
          )
          ..restore();
      }
    } else {
      canvas.drawAtlas(
        atlas,
        _transforms,
        _sources,
        _colors,
        blendMode ?? defaultBlendMode,
        cullRect,
        paint,
      );
    }
  }

I couldn't produce the artifacts anymore.

2022-01-01.19-23-46.mp4 Would be cool if someone else could confirm this, because I am not 100% if I am not seeing ghosts after spending to long on this problem.

Hope this helps!

I appreciate the efforts you have taken, good job. I tested with paint..isAntiAlias = false it worked in mine too. However, I was working on a latest version of tile image which I made by editing in photoshop which I haven't uploaded to github yet. In that newer version of game even though I made paint..isAntiAlias = false and tested the problem persists, sadly there are white vertical lines.

https://user-images.githubusercontent.com/52130837/147858738-48e7a634-2b75-4b59-b244-4f0e65e49e38.mp4

For the above new version I haven't added extrusion to the tile image(tile width is 64). Well, for the one that you have tested there I think I have added extrusion of 2 or 1 pixels(not sure), because in the tmx file the tile width and height is 66.

So, I think the actual problem is not fully solved. But you have found a work around to solve the problem @felithium 👏👏👏

[jacobranjit]

Hey, I might have found a solution. After spending hours trying different solutions to find a workaround (which I actually did aswell) I found out that this problem does not appear when you exactly use canvas.drawRectImage AND set the isAntialias of the Paint instance to false. If you for example use canvas.drawImage it simply does not matter if isAntialias is true or false, the artifacts will eventually appear. I have no clue why that is...

I fixed the game of @Jcupzz by stepping in the source of Flame where the TiledMap is painted. And it does use canvas.drawRectImage, but isAntialias is not set, so by default true, so I set it to false manually.

This is the file but the latest version is slightly different than the version of @Jcupzz: https://github.com/flame-engine/flame/blob/main/packages/flame/lib/src/sprite_batch.dart

void render(
    Canvas canvas, {
    BlendMode? blendMode,
    Rect? cullRect,
    Paint? paint,
  }) {
    paint ??= Paint();

    if (kIsWeb) {
      for (final batchItem in _batchItems) {
        paint..blendMode = blendMode ?? paint.blendMode;

        canvas
          ..save()
          ..transform(batchItem.matrix.storage)
          ..drawRect(batchItem.destination, batchItem.paint)
          ..drawImageRect(
            atlas,
            batchItem.source,
            batchItem.destination,
            paint..isAntiAlias = false, // i changed this
          )
          ..restore();
      }
    } else {
      canvas.drawAtlas(
        atlas,
        _transforms,
        _sources,
        _colors,
        blendMode ?? defaultBlendMode,
        cullRect,
        paint,
      );
    }
  }

I couldn't produce the artifacts anymore.

2022-01-01.19-23-46.mp4 Would be cool if someone else could confirm this, because I am not 100% if I am not seeing ghosts after spending to long on this problem.

Hope this helps!

I also discovered this bug some time ago... I wanted to make a tile based game with Flutter / Flame, but can't do it because of this. I opened issues on this in the Flutter (flutter/flutter#74127) and Flame #637 repo some time ago with no success. There is also a big issue (flutter/flutter#14288) on this in the flutter repo which was opened in 2018 but is not resolved yet...

I think it hast to do with the methods canvas.scale and canvas.translate. Every plattform even Android is affected (just scale the canvas by something like 1.32). It just happens that certain screen sizes with the right scaling dont produce this bug. For example when I move Mario in the game of @Jcupzz I have no problems only when i resize the browser windows from full screen. (I have a 4k monitor)

It would be cool, if we find a fix / workaround for this, because a friend of mine an me really want to make a tile based game with Flame.

The following is a minimal example to reproduce this (resize the window if it doesn't happen):

import 'package:flame/game.dart';
import 'package:flutter/material.dart';

void main() => runApp(GameWidget(game: MyGame()));

class MyGame extends FlameGame {
  Paint p = Paint()..color = Colors.blue;

  @override
  void render(Canvas canvas) {
    super.render(canvas);

    int tilesInX = 16;
    int tilesInY = 16;
    double tileSize = 16;

    canvas.scale(size.x / (tilesInX * tileSize));

    for (int x = 0; x < tilesInX; x++) {
      for (int y = 0; y < tilesInY; y++) {
        canvas.drawRect(
          Rect.fromLTWH(
            x * tileSize,
            y * tileSize,
            tileSize,
            tileSize,
          ),
          p,
        );
      }
    }
  }
}

I also tested this with paint..isAntiAlias = false but didn't work. So I think maybe in mario it worked because of the extrusion of 1 or 2 pixels( I don't remember it correctly)

[natebot13]

This could also have to do with rendering pixel art at non integer scales. Tiles that are, for example, 16 pixels wide, aren't going to render very well scaled to 17 pixels wide. This is a common issue with pixel art. It can be somewhat fixed by what's been discussed above, i.e. using antialiasing, but this subtly blends colors together, somewhat ruining the perfectness of pixel art. I've instead fixed it in my game by ensuring everything is positioned and scaled at integer increments. Crucially, I made my own copy of FixedResolutionViewport and modified it to only scale at integer increments. This works for my pixel perfect game but might not be suited for every game.

[st-pasha]

Crucially, I made my own copy of FixedResolutionViewport and modified it to only scale at integer increments.

Interesting, so you restrict the camera zoom level to integers (1, 2, 3, ...)? And the viewfinder's position to integer coordinates as well? And same for all the components?

[natebot13]

Interesting, so you restrict the camera zoom level to integers (1, 2, 3, ...)? And the viewfinder's position to integer coordinates as well? And same for all the components?

Yes, everything is stuck to an integer grid and specifically I ceil the scale of the viewport. This way it's locked to an integer scale slightly larger than the screen to fill it rather than flooring or rounding. This also means I can't take advantage of the built-in camera smoothing since I have to jump from pixel to pixel. I may figure out a way around that but haven't gotten there yet.

As a side note, I'm using flame_oxygen, which means I think I have more control to make my systems operate at integers only.

[st-pasha]

Interesting, what about component rendering -- how do you ensure that a component renders at the integer screen coordinates only?

[natebot13]

I just use a rounded version of the position component before rendering, which locks it to the pixel grid. Even if the viewport is scaled up, the sprites scale and lock to the pixel grid, keeping all the sprites consistent, and the whole scene like a single pixel art image.

[jacobranjit]

Crucially, I made my own copy of FixedResolutionViewport and modified it to only scale at integer increments.

Is there any sample code that you can share?

[natebot13]

Sure, it's a small change. I copied the full FixedResolutionViewport class from viewport.dart to a file in my project and in resize added a .ceil().toDouble() to the _scale:

157,160c55,61
<     _scale = math.min(
<       canvasSize!.x / effectiveSize.x,
<       canvasSize!.y / effectiveSize.y,
<     );
---
>     _scale = math
>         .min(
>           canvasSize!.x / effectiveSize.x,
>           canvasSize!.y / effectiveSize.y,
>         )
>         .ceil()
>         .toDouble();

I also round() the _scaledSize in order to ensure the viewport sits on an exact pixel, like this:

*** 165,168 ****
      _resizeOffset
        ..setFrom(canvasSize!)
        ..sub(_scaledSize)
<       ..scale(0.5);
--- 66,70 ----
      _resizeOffset
        ..setFrom(canvasSize!)
        ..sub(_scaledSize)
>       ..scale(0.5)
>       ..round();

You can replace either of those with ceil, floor, or round for your needs, but I chose ceil for _scale so that the scene would scale outside of the window to fill the screen more. Of course that cuts off some of my game world but that's fine for my game. I round _resizeOffset to match all my other entities which are also rounding their coordinates.

Then of course create and apply your custom viewport to the camera. Camera and Viewport

[spydon]

Do you want to PR that viewport @natebot13? It seems like it could be useful to others too.

[ryokuchayurai]

Thank you for the promising solutions.

I added the filterQuality setting in addition to paint..isAntiAlias = false and it improved mine.

void render(
    Canvas canvas, {
    BlendMode? blendMode,
    Rect? cullRect,
    Paint? paint,
  }) {
    paint ??= Paint();

    if (kIsWeb) {
      for (final batchItem in _batchItems) {
        paint..blendMode = blendMode ?? paint.blendMode;

        canvas
          ..save()
          ..transform(batchItem.matrix.storage)
          ..drawRect(batchItem.destination, batchItem.paint)
          ..drawImageRect(
            atlas,
            batchItem.source,
            batchItem.destination,
            paint..isAntiAlias = false
              ..filterQuality = FilterQuality.low, // i changed this
          )
          ..restore();
      }
    } else {
      canvas.drawAtlas(
        atlas,
        _transforms,
        _sources,
        _colors,
        blendMode ?? defaultBlendMode,
        cullRect,
        paint,
      );
    }
  }

[ryokuchayurai]

I have found that a modification to the RenderableTiledMap improves the situation.

@@ -135,7 +137,7 @@ class RenderableTiledMap {
     await Future.forEach(map.tiledImages(), (TiledImage img) async {
       final src = img.source;
       if (src != null) {
-        result[src] = await SpriteBatch.load(src);
+        result[src] = await SpriteBatch.load(src, useAtlas: kIsWeb ? false: true);
       }
     });
 
@@ -192,10 +194,14 @@ class RenderableTiledMap {
     });
   }
 
+  final Paint _paint = Paint()
+    ..isAntiAlias = false
+    ..filterQuality = FilterQuality.low;
+
   /// Render [batchesByLayer] that compose this tile map.
   void render(Canvas c) {
     batchesByLayer.forEach((batchMap) {
-      batchMap.forEach((_, batch) => batch.render(c));
+      batchMap.forEach((_, batch) => batch.render(c, paint: kIsWeb ? _paint: null));
     });
   }

I do not know if this modification is appropriate....

[natebot13]

I recently changed monitors and this issue cropped up for me again, despite my previous rounding fixes I had made. I then figured out that that this issue is actually a consequence of Flutter's feature of using the devicePixelRatio to keep the size of widgets the same across all devices. This is a problem because even if you round your draw coordinates, where the sprite actually lands on the screen might be between two physical pixels. I'm looking into a solution that takes the pixel ratio into account to hopefully place all the pixels onto physical pixels precisely.

[spydon]

Not really a solution, but if you run with Impeller this issue shouldn't occur.

[spydon]

Here's a great article by @erickzanardo that explains how to work around this issue: https://verygood.ventures/blog/solving-super-dashs-rendering-challenges-eliminating-ghost-lines-for-a-seamless-gaming-experience

[benni-tec]

@spydon Thank you, for linking the article. I was running into the same problem and will probably endup doing this in the short term. However it seems that is a persisting issue and a Viewport that scales to the device pixel boundaries seems to me like a more sustainable solution.

I looked into implementing #2810 however I was not able to. I managed to get the lines to be static (instead of changing when moving) but I was unable to replicate the effect that @natebot13 was able to achieve.

Is there any documentation on how the Viewport/Viewfinder work together and why these classes all inherit a resolution, and how the ScaleProvider affects the cameras behaviour? Related to this I have also found it difficult to convert screen coordinates into world coordinates! However I could not understand what is actually happening from the source code alone and have not found additonal resources.

I am happy to implement #2810 myself if there is further documentation available, otherwise any help would be greatly appreciated!

[spydon]

Is there any documentation on how the Viewport/Viewfinder work together

If you've already read the docs about it on our docs page I can recommend reading the dartdocs that accompany the code, they are pretty good.

and why these classes all inherit a resolution, and how the ScaleProvider affects the cameras behaviour?

The scale for the viewfinder is the zoom and the scale for the viewport is used to set up things like the FixedResolutionViewport.

Related to this I have also found it difficult to convert screen coordinates into world coordinates!

Just use globalToLocal and localToGlobal: https://github.com/flame-engine/flame/blob/main/packages%2Fflame%2Flib%2Fsrc%2Fcamera%2Fcamera_component.dart#L209

[benni-tec]

If you've already read the docs about it on our docs page I can recommend reading the dartdocs that accompany the code, they are pretty good.

I think I already looked at the code and did not find dartdocs on these classes, however this was a few weeks ago. I will look into it again!

The scale for the viewfinder is the zoom and the scale for the viewport is used to set up things like the FixedResolutionViewport.

That makes sense! Thank you :)

Just use globalToLocal and localToGlobal: https://github.com/flame-engine/flame/blob/main/packages%2Fflame%2Flib%2Fsrc%2Fcamera%2Fcamera_component.dart#L209

I think I did however the values did not match what I expected, I was particulary confused because these can be called on multiple objects. Did I understand correctly that I first need to translate them from screen to global coordinates using the camera and then from global to local using the World component?

[benni-tec]

Just took a brief look at the code again: While the default viewports do have dartdocs explaining what they do, they do not have dartdocs/comments explaining how they do it.

I tried basing a device-pixel integer viewport on the FixedResolutionViewPort and the old implementation I believe, but did not really understand what is happening, esspecially what the class it self, it's super-class and the ScaleProvider class change regarding the viewports behaviour.

I will look into this further after the holidays 🎅

[spydon]

I think I did however the values did not match what I expected, I was particulary confused because these can be called on multiple objects. Did I understand correctly that I first need to translate them from screen to global coordinates using the camera and then from global to local using the World component?

No, you only have to do it on the CameraComponent and it will bring it straight to world coordinates, since the world doesn't modify the coordinates, it is only observed through the camera.

[benni-tec]

No, you only have to do it on the CameraComponent and it will bring it straight to world coordinates, since the world doesn't modify the coordinates, it is only observed through the camera.

Ah just had a quick look at my code, turns out I was only using the viewfinder to convert from canvas to world coordinates, I'm guessing thats where the discrepancy came from. I will test it once I'm back at work.

Thank you for the quick response, eventhough this is off-topic for this issue 😄

[spydon]

Ah true, dartdocs are for "what it does" not "how it does it".

Thank you for the quick response, eventhough this is off-topic for this issue 😄

Np, I can recommend joining our Discord server if you're not there already btw.

[benni-tec]

So I tried again to fix this issue, firstly I added margins and spacing. That seemed to make the lines "smoother" or lighter, but they are still there. However they are now static and horizontal!

I tried to implement the DeviceIntegerViewport without much luck:

class DeviceIntegerViewport extends Viewport implements ReadOnlyScaleProvider {
  static FlutterView _firstView() =>
      f.WidgetsBinding.instance.platformDispatcher.views.first;

  bool clipping;

  DeviceIntegerViewport({this.clipping = true});

  double _ratio = 0;

  @override
  Vector2 size = Vector2.zero();
  
  @override
  Vector2 get scale => Vector2.all(1 / _ratio);

  @override
  void onLoad() => _resize();

  @override
  void onViewportResize() => _resize();

  void _resize() {
    final _size = _firstView().physicalSize.toVector2();
    if (_size == size) return;

    size = _size;
    _ratio = _firstView().devicePixelRatio;
    position.round();
  }

  @override
  void clip(Canvas canvas) {
    if (clipping) canvas.clipRect(size.toRect());
  }
}

A side from not showing any visible effect it also does not change size properly but I do not understand why, when resizing the browser window it seems to change size at all (causing clipping when enlarging and not moving the centered player to the center). However as far as I understand this should work, what am I missing?