Graphics error on Pixel 6/6S

[everseen83]

**** Development environment ****

• AIR SDK : 33.1.1.575
• Starling Framework : 2.6
• android:targetSdkVersion="30"
• android:hardwareAccelerated="true"
• <containsVideo>true</containsVideo>

Error reports like the attached image are coming from Pixel 6/6S users. The game runs OK, but flickering occurs. The distorted image is an Atlas file from the game in PNG format(not ATF). It is not an Android 12 issue. It may have something to do with the chipset of the Pixel 6/6S. The Pixel 6/6S is not sold in the country, where our company is located. So we cannot reproduce the bug ourselves.

Any help in resolving this problem would be appreciated.

[everseen83]

As a result of testing, these errors only appear in "armv7". I hope this helps in solving the problem. @ajwfrost

[marchbold]

We are seeing some glitches on a pixel 6 pro as well, demonstrated in the video below.

Our tests are unfortunately contradicting the above, we are encountering them with armv8. armv7 builds don't seem to have the issue.

https://user-images.githubusercontent.com/442356/144374684-877da754-b1b0-4c34-8e51-d17e9feae56e.mov

• AIR SDK 33.1.1.686
• AAB and APK (armv8) targets are exhibiting the issue

[ajwfrost]

So from what I'd previously heard, the glitches were somewhat random in occurrence, but the above video suggests there's something about a particular element here which is causing a problem. @marchbold is that reproducible if you run the application again? or just sometimes? And do you have any info about the different sizes of textures that were being run there?

There's perhaps something with this particular phone GPU or driver that we need to work around, but getting a hint of how to reproduce the problem would be very handy..

thanks

[marchbold]

It's completely reproducible, I'll have to track down some details on the texture sizes tomorrow.

[ajwfrost]

Thanks @marchbold - if it's something we could try to reproduce here too it would be good - would it be possible to get an "apk-debug" build of it plus instructions?

@everseen83 likewise, if you are seeing this reproducing then if you can do a build with "apk-debug" target and send it to us, we can load it up with a debug-enabled opengl layer and see what happens...

thanks

[kidgamez]

I am also getting this error with Pixel 6! Using latest SDK (33.1.1.686), Starling 2.7

https://youtu.be/h48WC7v6MwI

https://youtu.be/cPaTnSclL4Y

[ajwfrost]

Hi all

We do seem to have a problem that's restricted to just Pixel 6 devices... if folk are able to report this to Google it would be useful, we have another company that are already talking to them about it and there's a previous known issue which doesn't sound exactly the same, but may indicate some driver (or even hardware) issue on that phone range.

Plus if there are any reproducible (and as simple as possible) examples, if we can get the apps built with apk-debug mode (or aab-debug), we can look at introducing some OpenGL output which allows us to turn the app into a canned "pure OpenGL" application i.e. nothing to do with AIR, just a series of OpenGL instructions which would then help us/Google work out what's going wrong.

thanks

[marchbold]

We tried reducing our texture atlases down to < 512 and we are still experiencing this issue. So there aren't any large textures in use. Draw calls are around 10 so shouldn't be an issue.

https://user-images.githubusercontent.com/442356/144971236-34415ec2-78a4-4afc-ae9a-fe17ef61b5a4.mov

I'll email you a debug aab.

[JohnBlackburne]

Google's supersized December update for Pixel phones includes a ton of fixes for the Pixel 6 and 6 Pro But you'll have to wait a little while longer to install it

Google is also fixing some display issues plaguing the Pixel 6 since its launch. We've seen countless user reports of screen flickering whenever the phone's brightness is adjusted or the screen is locked, something that this new software should rectify

[shohei909]

It seems to occur when the number of vertices in a single Mesh exceeds 99.

In our case, we were able to avoid this problem by setting starling.display.MeshBatch.MAX_NUM_VERTICES to 99 and avoiding the use of starling.display.Image.tileGrid.

[marchbold]

@shohei909 Great tip, just tried this in our app and appears to have resolved the main glitches we are seeing. We still have some usages of the tile grid but setting the MAX_NUM_VERTICES to 99 seemed to have resolved all the glitches we were seeing.

@ajwfrost @PrimaryFeather Does this give you any clues as to what the issue might be?

[ajwfrost]

We had a quick look at this MAX_NUM_VERTICES thing, it's limiting the size of the vertex buffers but I presume there's no error codes coming back from the device when trying to create/upload the buffers hence no clue that it's not working like it does on other phones. We can see if there's any metrics that we can glean from the OpenGL APIs, or if not we can put this into a "known restrictions" set of features in order to limit the buffer sizes to prevent the issue. But of course, this would require a change in Starling to make that MAX_NUM_VERTICES be a dynamic property read somehow from Stage3D APIs or similar...

(I'm assuming that it would be fairly catastrophic for the application if we just returned/threw an error if a higher number of vertices were sent into the Stage3D API...)

@shohei909 thanks for finding this, are you also seeing the glitching if you have the max vertices limited but are still using tilegrid?

thanks

[PrimaryFeather]

MAX_NUM_VERTICES in Starling is used while meshes are batched together during the render process. When a MeshBatch would exceed this number of vertices when adding a new mesh, a new MeshBatch is created instead (and the previous one is drawn).

There are situations when this limit isn't enforced, though – e.g. when using Image.tileGrid. That grid can potentially require more vertices than MAX_NUM_VERTICES, but Starling won't complain but attempt to send it to the GPU anyway. (As far as I remember, this would lead to an exception anyway on normal devices).

But of course, this would require a change in Starling to make that MAX_NUM_VERTICES be a dynamic property read somehow from Stage3D APIs or similar...

I'd gladly make that change, of course! When there is an API I can query about the maximum number of vertices / bytes (or whatever) per vertex buffer, I can exchange the hard-coded value with that call. Just let me know if and when this API is available, @ajwfrost !

[ajwfrost]

Great thanks @PrimaryFeather - that explains the issue with the tile grid that could cause a lot of vertices to be created...

So what we're saying is that the Pixel 6 seems to have an issue when we're calling either Context3D.createVertexBuffer() or VertexBuffer3D.uploadFromByteArray().

We may need to check how the Pixel 6 is being treated, whether we're using a Vertex Buffer Object or not.

We are looking at the device to see the number of vertex attribute arrays that are available (i.e. from GL_MAX_VERTEX_ATTRIBS) but it doesn't look like there are actually any methods to get the maximum size of the memory available for vertex data... other than to check for "out of memory" scenarios. If it's using a vertex buffer object then we should find out about this and throw an error when creating the vertex buffer, but if it isn't then there may just be an error during rendering itself and I don't know whether that's picked up...

I think we need to do some testing/tracing of the behaviour on a Pixel 6 device to see if we can detect this problem programmatically and then try working around it somehow....

thanks

[kidgamez]

Is there anything we can do as a temporary fix for Google Pixel 6 users?

[marchbold]

@kidgamez Did you try setting the MAX_NUM_VERTICES to 99? This resolved it enough for us.

[RossD20Studios]

I just got a Pixel 6 today and was dismayed to discover this exact same problem in the game I'm developing. Here are some example images and videos: Initially I suspected the texture atlas was bleeding through...

And this screenshot confirms that.

Here's a video of the flicker: https://youtu.be/vdADyjwD6TE

Here is after I applied the MAX_NUM_VERTICES = 99 fix. That helped, but unfortunately, did not completely solve the problem. My game also uses particle effects to create a "mist" around unexplored areas. As I move my character towards these particles, the artifacts return with a vengeance. https://www.youtube.com/watch?v=5hGKqKK9JVo

I hope that we can do better than a temporary workaround for Google's flagship phone - and I am willing to offer my assistance however I can to help my fellow developers test and resolve this issue now that I have the Pixel 6.

[RossD20Studios]

I spoke with @PrimaryFeather about this and did some further testing to help determine what the maximum limitations of the Pixel 6’s current graphics drivers are on the Starling/AIR engine. Here’s the findings:

First, while setting MAX_NUM_VERTICES == 99 does remove the glitching problem, making this change also has an effect on draw calls. I imagine these results will vary for each developer’s project, but for my game, the impact was pretty dramatic:

When MAX_NUM_VERTICES == DEFAULT, I’m getting 8 draw calls with my character standing at a particular spot in a forest of mesh animated trees. When I set MAX_NUM_VERTICES = 99, my draw calls standing in the same spot jumps from 8 to 50. More draw calls means the GPU is working harder, the phone is getting hotter, and the battery is draining faster. Next, I tested ranges of values until I determined an exact number where the glitches start to occur and how the draw calls are effected. Here at the results:

DRAW CALLS AND GLITCHES PER MAX_NUM_VERTICES: At 99, 50 draw calls (no glitching) At 125, 45 draw calls (no glitching) At 150, 37 draw calls (no glitching) At 300, 21 draw calls (no glitching) At 500, 15 draw calls (no glitching) At 502, 15 draw calls (no glitching) At 503, 15 draw calls (no glitching) At 504, 15 draw calls (Glitching) At 508, 15 draw calls (Glitching) At 513, 15 draw calls (Glitching) At 525, 15 draw calls (Glitching) At 550, 14 draws calls (Glitching) At 600, 14 draw calls (Glitching)

The one nice thing about this bug is that it doesn’t appear to have "degrees" of glitching. The artifacts either appear or don’t, and I observed that the glitching starts happening at MAX_NUM_VERTICES == 504, making 503 the maximum "safe" limit for Pixel 6.

Second, changing the MAX_NUM_VERTICES value will NOT fix the artifacts caused by the particles system. To fix this, you need to change the MAX_NUM_PARTICLES in the ParticleSystem class. Again, I tested ranges of values until I determined that a setting of 125 is the maximum safe limit. A value of 126 or more will result in the artifacts returning when particle systems are being rendered.

Note, however, that there is no draw call impact to changing the MAX_NUM_PARTICLES because this actually reduces the number of particles. Instead, the impact is a visual downgrade (the particle systems will look more sparse) - and how that effects each developer’s game will vary based on the design of your particle emitter FX.

To summarize, I would recommend changing the temporary fix until a better solution/driver fix is made to:

MAX_NUM_VERTICES = 503 MAX_NUM_PARTICLES = 125

Is there any way we can detect the Pixel 6 as the active device so that this change can be made dynamically so that performance and visual quality are not affected for other devices?

[ajwfrost]

Hi @RossD20Studios - thanks for that analysis! Useful to have a better set of values here... I think there's some concern that there are other constructs that may still cause problems where the number of vertices passed into the vertex buffer are not limited by those variables, but that sounds like a good approach and if it works in practice for you then it probably covers the necessary stuff here...

We had been trying to look at whether we can programmatically detect the limits. There doesn't appear to be any OpenGL ES API that works here to give us a device limit, and certainly no AS3 API that could be used to query it currently (although a VertexBuffer has a maximum number of vertices of 65535, and there's also a memory limit of 256MB...)

There are already a number of device-specific (or rather chipset-specific) workarounds within the rendering engine, so we can perhaps add that to the runtime, but again it means a problem in trying to get that information up to Starling. So the alternative is to just expose the fact that it's a Pixel 6 device .. which we may be able to do via the existing capabilities object or similar. Let me check.....

thanks

[RossD20Studios]

You're welcome, and of course I agree with you and would much prefer a true solution to this issue. I wasn't initially aware that changing these variables impacted draw calls (which could adversely affect the experience for users on other devices) and wanted to let others know how it could potentially impact their apps as well. My interim plan is to add a setting to my game options that sets/removes these variable limits and either instruct Pixel 6 users to enable it or make it the default and advice anyone experiencing performance issues to turn it off. I very much appreciate that you and your team on top of this and look forward to your updates, thank you!

[GeneralVimes]

I also got report like this for my game: https://youtu.be/WLIMkJUFIiA (edited it making 10 times slower) Many thanks to Starling forum members who suggested to check this thread!

For my game, which has many buttons and informers with text, as I set MAX_NUM_VERTICES to 99 the draw calls increase from 1 to 130. I think I shouldn't do this for now. Looking forward to seeing what can be done.

Another thought was to make the game unavailable for Pixel 6 (it's around 0.03% of the installers), but I'm wary if this might hurt the game's discoverability in Google Play (just, as Pixel 6 is produced by Google, they must be interested in the game to be playable on their devices). Maybe Google can just fix it on their side, because it's ridiculous when the top-notch device can't handle 65k vertices, and can make at most 500. Let's see.

Offtop: @RossD20Studios really glad to see you here and to know that Summoner's Fate is made with AIR as well! We met at several Big Indie Pitches online. Why don't you submit your game to airsdk.dev showcase?

[RossD20Studios]

@GeneralVimes Hi Alexey, good to see you here as well! I wasn't aware there was an airsdk.dev showcase, thanks for letting me know, I will submit Summoners Fate :)

[RossD20Studios]

@GeneralVimes How do you add your application to the showcase? I clicked the link to "Add your application" from this page here: https://airsdk.dev/showcase but it took me to a Github page saying I need to fork the repository to make changes. Is that intended?

[GeneralVimes]

@GeneralVimes How do you add your application to the showcase?

Yes, this is how I added my games there:

1. Fork the repository
2. Cloned this repository to my computer with Github Desktop (that was more convenient for me)
3. Added games - edited airsdk.dev\src\data\applications.tsx according to the template inside and added a 800x400 images of my games into airsdk.dev\src\data\showcase\ folder
4. Pushed my changes to the origin from Github Desktop
5. Then I went to my fork of airsdk.dev on my Guthub account and clicked the link showing that 'm 1 commit ahead of the airsdk:main
6. crated a pull request
7. @marchbold approved it

Maybe I wrote a too detailed and some steps might be obvious, just the last time I contributed to a group project on Github in 2017, so, I had to recall how it's done, and perhaps this can be useful as instruction to other AIR game developers

[RossD20Studios]

Thanks for the instructions @GeneralVimes ! I think I followed your steps correctly, just waiting for @marchbold to approve :)

[marchbold]

@RossD20Studios I don't see a pull request on the airsdk repo? You may not have quite done that last part correctly. You need to do a pr back to the main airsdk repo if you want it merged in and displayed on the site.

[RossD20Studios]

Thanks @marchbold - You should now see two pull requests, one for the image, another for the JSON addition.

[marchbold]

@RossD20Studios Ah okay, I normally wouldn't accept that as you should package everything into one PR but I'll have a look over it later today and see what I can do.

[GeneralVimes]

As for the glitch, I'm going to try such workaround:

			if (Capabilities.os.indexOf("Pixel 6")!=-1){
				MeshBatch.MAX_NUM_VERTICES = 400;
			}

Because as one of the players showed me, Capabilities.os contains "Pixel 6" substring. Will tell you how it worked

Also, to do this, I'll need to change MeshBatch: the public static const MAX_NUM_VERTICES must become public static var MAX_NUM_VERTICES @PrimaryFeather what do you think, might this change (const to var) affect performance for non-Pixel 6 users?

[RossD20Studios]

@marchbold - I saw you approved the showcase changes. It looks like the image one worked, but the update to the JSON failed. Apparently the build test failed with "Process completed with exit code 1." although I didn't see this on my end when I submitted the pull request. I checked over the code again and I can't find any errors - I literally copied and pasted the template and replaced the values to match existing structure.

@GeneralVimes That's cool! I didn't know the capabilities can detect Pixel 6. I had previously resolved this program by adding an optional setting in my menu called "Pixel 6" fix that enabled setting the MAX_NUM_VERTICES.

In my testing, I didn't observe any performance cost to changing the MAX_NUM_VERTICES from const to var, but, I did notice differences in performance based on the values you set for MAX_NUM_VERTICES. Ideally, you want to find the highest number of MAX_NUM_VERTICES you can set without observing any glitching patterns, as the lower this number is, the more draw calls are required.

These are the safe thresholds I found (I don't know if this would differ from app-to-app):

MAX_NUM_VERTICES = 503 MAX_NUM_PARTICLES = 125

Also not that if you use any particle systems, you'll also want to set that value as the glitch will still happen even if you set the MAX_NUM_VERTICES.

Let me know if that capabilities check works as expected, I'll implement that in my game, too!