SafeArea using Screen.mainScreen.safeArea Reports Incorrect on Select Android Devices

[RossD20Studios]

Problem Description

I am using AIR's Native safeArea via Screen.mainScreen.safeArea in order to position my game's UI elements away from the camera/notch area of the device.

I primarily test builds on Google Pixel devices (where this feature is working correctly in AIR 51.1.4) but users on select Android devices (Samsung Galaxy S10, VivoY3 and Infinix X680B) have reported issues where the safeArea does not report correctly, resulting in the game positioning UI elements off screen.

I recently tested my game using AIR 51.2.2, but this build worsened the problem by breaking the previous functionality that was working on my Google Pixel devices.

I'm going to share screenshots, my App descriptor XML settings, and some code illustrating how the UI is setup.

Steps to Reproduce

Relevant Parts of App Descriptor XML

<initialWindow>
        <content>main_android.swf</content>
        <systemChrome>standard</systemChrome>
        <transparent>false</transparent>
        <visible>true</visible>
        <fullScreen>true</fullScreen>
        <renderMode>direct</renderMode>
        <autoOrients>true</autoOrients>
        <depthAndStencil>true</depthAndStencil>
        <orientationAnimation>none</orientationAnimation>
    </initialWindow>

<android>
        <manifestAdditions><![CDATA[
        <manifest android:installLocation="auto">
            <uses-sdk android:minSdkVersion="24" android:targetSdkVersion="35" />
            <uses-feature
                android:name="android.hardware.type.pc"
                android:required="false" />
            <uses-permission android:name="android.permission.INTERNET"/>
            <uses-permission android:name="com.android.vending.BILLING"/>
            <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
            <uses-permission android:name="com.google.android.providers.gsf.permission.READ_GSERVICES"/>

<application android:hardwareAccelerated="true" android:appComponentFactory="androidx.core.app.CoreComponentFactory">
                <meta-data android:name="android.max_aspect" android:value="3.5" />
                <meta-data android:name="android.notch_support" android:value="true"/>

I will show the screenshots and code examples below in separate comments.

Known Workarounds

None

[RossD20Studios]

Let's start with the Google Pixel 5 (Primary development device) - Here's what the phone looks like:

Here's how the screen should look. These correct results are produced in AIR 51.1.4

correctTitleScreenPixel5Portrait

correctBattleScreenPixel5Portrait

correctBattleScreenPixel5PortraitUpsideDown Notice here that there is a black bar that appears below the HUD. This is something that I programmatically added because having open gameplay area appear below a static HUD looks strange. This way, if the user's phone is upside down, it appears to fully cover the camera notch with the black bar and the game looks visually correct.

correctBattleScreenPixel5Landscape

correctExplorationHUDPixelLandscape

With AIR 51.2.2, the UI is displayed incorrectly on the exact same Pixel 5 device (no code changes):

incorrectTitleScreenPixel5Portrait Notice here how the buttons on the lower right have been shifted inward.

incorrectBattleScreenPixel5Portrait Notice the cards and UI at the bottom are shifted off center to the left.

incorrectBattleScreenPixel5Landscape Notice the cards and UI on the left are shifted off center to the bottom.

incorrectExplorationHUDPixelLandscape Notice the entire HUD has been shifted down into the gameplay area, presumably because it's detecting camera space on the long edge of the device.

[RossD20Studios]

Some of my player's were kind enough to share screenshots of the issue on their devices. Here is a screenshot of the problem illustrated on VivoY3 in AIR 51.1.4 :

Here is the issue illustrated in AIR 51.1.4 on a Infinix X680B:

Notice in both of these images that the UI is shifted by exactly the space of the black bar generated at the top of the device. I believe this suggests that safeArea on this device in portrait mode (vertically) is actually reporting in reverse. The black bar should always appear over the camera BUT only if the phone is being held upside down as I illustrated in the correct screenshots for the Pixel 5.

[RossD20Studios]

So far I have kept the game in AIR 51.1.4, because switching to AIR 51.2.2 creates a worser problem (on Pixel devices) than the potential solution for the devices experiencing the safeArea inaccuracy on AIR 51.1.4.

A new solution for detecting safeArea on select Android devices ( Samsung Galaxy S10, VivoY3 and Infinix X680B ) without disrupting the safeArea already defined for other devices (like Google's Pixel series) is needed in a newer version of AIR 51.2

[RossD20Studios]

Here is the code I use to generate and place the black bar (scrim) to cover up the camera notch area if the phone is turned upside down:

If the phone is held right side up, this code will result in any UI at the top of the screen being shifted below the camera notch area.

iPad's had their own funky behavior, so I created a custom function to deal with these specifically:

public static function isIpad(platform:String, screenW:int, screenH:int):Boolean{
	var w:int = Math.max(screenW, screenH);
	var h:int = Math.min(screenW, screenH);
	return (platform == iOS && w/h <= 1.45);
}

[marchbold]

Hey @RossD20Studios Would you be up to trying my new Display extension for this? It's still under development (I'm pulling all the display elements out of the Application extension and rewriting the API to be more consistent).

Currently have the Android safe insets implemented so should fix your issues here I believe:

The insets are like margins on each side of the screen.

var insets:Insets = Display.instance.windowInsets.getSafeArea();

insets.left; // amount to inset ui elements from the left
insets.top; // amount to inset ui elements from the top

https://drive.google.com/file/d/1mBQSnxbjsDKqppuAetMNHrmAvz2YeFQP/view?usp=sharing

[ajwfrost]

Hi @RossD20Studios

Thanks for all the details and screenshots. Can I check though: the screenshots from the end users that are showing the black bar at the top, with the whole UI then shifted down -> they are still using 51.1.4.1? Do you know what Android version those were running - I think we have an issue caused by the changes in Android 15 that's mentioned in #3949. Plus just to check - when the UI had the black bar and the shifted UI - were those devices being held upside down i.e. camera notch at the bottom?

We had implemented an update in 51.2.2.4 to account for this but it appears this is causing some other issues per your screenshots from the above Pixel tests.

So we'll need to update the safeArea implementation to ensure that this is correct for all the devices/orientations, but that doesn't explain what happened to those customer devices where there's a black bar appearing whilst the 'height' of the game display appears correct, i.e. it's an additional y-offset that's being added. My understanding of the changes in Android was just that the APIs we were using for safe area wouldn't include the status bar areas now i.e. we're just checking for notch/cut-out areas - which means, nothing should have changed, and worst case is the status bars would be displayed on top of the app where you thought it was safe to show crucial content.

For the black bar to appear at the top, per the customer images and your code, it implies layoutRectCenter.height is equal to _scrim.height i.e. the y-value of the bar is zero. And for _scrim.height to match layoutRectCenter.height that implies that safeArea.height is zero ..? Which is definitely wrong ..! but how do you determine how much to shift your main UI down by? I'm actually thinking it may be more likely that the black bar is added by Android .. although you do have the fullScreen value set to true. Curious..

Have you found any devices where you can reproduce this with the 51.1.4 build yourself? Or do you have one of these folk who might be able to try something out and capture some metrics?

thanks

[marchbold]

@ajwfrost We have a few devices here and even a standard pixel device is producing issues. Eg with fullscreen false and a simple app you can see the purple lines are what air is reporting and red is what I'm returning in the Display ANE I'm working on.

This is just the device in portrait.

• AIR SDK 51.2.2.4
• Android 16

[ajwfrost]

Okay so we've found an issue in the code that was added in AIR SDK 51.2.2.4 where the 'top' and 'right' values are being incorrectly swapped.. which means that the above screenshot is showing a safe area that's cut in on the right hand side, whereas it should be on top. If you rotate the device to the left, then the safe area should cut in on that leftmost part correctly, with the top and right areas both being the same as the visible bounds now.

We can get that fix into 51.2.2.5 which will hopefully be out next week. (Side note .. the safe area only really works if you have full screen mode set to 'true'...)

I'm still unclear what's happening in the existing 51.1.4 version with these customers though, we'll have to check that a little further, I am assuming it's related to the Android update and our lack of handling for it..

thanks

[ajwfrost]

@RossD20Studios we may have worked out why the 51.1.4 version is going wrong for you, I think it's related to the targetSdkVersion="35" which I assume is a relatively recent addition? Can I check whether you have got a value in the app descriptor for <displayCutoutMode>? We got the same effect as in your screenshots (in portrait normal mode) when we were targeting API level 35 but without that tag; adding <displayCutoutMode>always</displayCutoutMode> then made it display correctly..

thanks

[RossD20Studios]

Thanks so much @marchbold and @ajwfrost for your timely response and super-level support!

@marchbold - Sure, I'm happy to test the extension! I'll give it a shot today and let you know how it works :) @marchbold 's screenshot above also appears to confirm the behavior I was seeing on my Pixel in AIR 51.2.2.4 as well, thanks for posting that!

@ajwfrost - I do include a displayCutoutMode tag that's currently living in the manifestAdditions section of my application descriptor (sorry I didn't include this in the notes above earlier, it was hiding at the very end of the descriptor). Here's the code block for that section:

</application>
        </manifest>
    ]]></manifestAdditions>
        <runtimeInBackgroundThread>true</runtimeInBackgroundThread>
        <displayCutoutMode>shortEdges</displayCutoutMode>
    </android>

Mine is using "shortEdges" though instead of "always". I can try changing that value, what's the difference between these options?

For the players I have who recently reported this issue in my AIR 51.1.4 version, here's what AIR SDK capabilities class reported back as their OS version (I'm not sure how to translate this into Android version though):

Samsung Galaxy S10: Android Linux 4.14.78-16886901-SM-G973U1, 1080x2280 DPI 420 VivoY3: Android Linux 4.9.117+-V1901A, 720x1544 DPI 320

To confirm, you're saying if I change to <displayCutoutMode>always</displayCutoutMode> then you'll get the correct safe area and screenshots as I was showing for my Pixel in AIR 51.1.4?

Thanks again!

[ajwfrost]

Hi

For the difference between the different cutout modes: https://developer.android.com/develop/ui/views/layout/display-cutout I'm not actually sure then whether this will make a difference or not, but we found we had the black bar at the top when we left this out (aka 'default') but the display was properly covered when we set it to 'always'. I guess we should have also tested 'short edges'!

So for us this worked, but I don't for sure know if it's the same reason that your display was being shifted down..

Plus - I'm not actually sure how to translate that version into a normal Android version code... is that the Capabilities.os value? I think we should see if we can get a better bit of information than that!! but the discrepancies in the numbers make me think they may not both be the latest one or two versions..

We'll do a bit more digging and testing..

thanks

[RossD20Studios]

@ajwfrost - It looks like my app descriptor is targeting SDK 35: <uses-sdk android:minSdkVersion="24" android:targetSdkVersion="35" /> and according to the documentation on android developer:

LAYOUT_IN_DISPLAY_CUTOUT_MODE_SHORT_EDGES: content renders into the cutout area in both portrait and landscape modes. Don't use for floating windows. If your app targets SDK 35, this is interpreted as ALWAYS for non-floating windows.

This suggests I should be getting the correct results even with short edges. I don't have a Galaxy S10 to confirm this, but I can try flipping the switch in my game's next patch and ask players if they can confirm whether or not the UI layout is corrected. If your team has access to the Galaxy S10 and can verify this change works, let me know. It'd be great to know that A) You're seeing black bars as shown above currently and B) Changing this flag indeed removes the black bars and positions UI correctly.

@marchbold - For your Display ANE, I wanted to confirm my understanding of how it works so I can use my existing UI logic. Let me know if the following assumption is true:

var insets:Insets = Display.instance.windowInsets.getSafeArea(); var safeArea:Rectangle = new Rectangle(insets.left, insets.top, insets.right-insets.left, insets.bottom-insets.top); //safeArea == Screen.mainScreen.safeArea (Assuming Screen.mainScreen.safeArea is accurate)

This would allow me to create a customizable class that could attach your ANE solution for Android (where the safeArea is currently broken) and maintain the current use of Screen.mainScreen.safe area for other platforms without depending on the ANE.

[ajwfrost]

Hi

We've not got an S10 but let me see whether we can find someone who does..

I agree with your assessment of the Android documentation, but I'm not convinced that this is actually what we see in practice!

thanks

[RossD20Studios]

Thanks @ajwfrost ! I'll probably push another patch today and I can try and test today as well.

@marchbold - I did some testing with your ANE. It looks like my original assumption above about the logic was incorrect, but I believe I now understand how to adapt it to have the same logical meaning as Screen.mainScreen.safeArea.

Here's what the insets class produces on my Pixel 5:

Here's what Screen.mainScreen.safeArea produces on my Pixel 5:

I used the following method to convert insets to safeArea rectangle:

public function get safeArea():Rectangle{
	if(displayServiceSupported){
		var insets:Insets = Display.instance.windowInsets.getSafeArea();
		var safeArea:Rectangle = Screen.mainScreen.bounds;
		safeArea.x = insets.left; safeArea.width -= insets.left;
		safeArea.y = insets.top; safeArea.height -= insets.top;
		safeArea.width -= insets.right;
		safeArea.height -= insets.bottom;
	}
	else{
		safeArea = Screen.mainScreen.safeArea;
	}
	return safeArea;
}

On my Pixel 5, this generates the same safeArea, but please let me know if that looks correct to you or if I made any bad assumptions.

With this, I can now use a DisplayService class that be added to specific platforms without creating dependencies for the ANE within the game's main source code. The Main application for each specific platform can override a getCustomSafe area method with the one generated by the service and also default back to the native Screen.mainScreen.safeArea if the ANE is not supported.

I tested your ANE on my Pixel 5 with both AIR 51.1.4 and AIR 51.2.2.4, and the results generated were both consistent and generated the correct UI layout :)

@ajwfrost - what would be the most helpful thing I can do for the next step? A) Release my next build in AIR 51.1.4 with the <displayCutoutMode>always</displayCutoutMode> enabled, and ask players who reported the layout issue if this change corrects the UI layout?

B) Release my next build in AIR 51.1.4 with @marchbold 's Display ANE enabled for Android and ask players who reported the layout issue if this change corrects the UI layout?

C) Same as B), but with AIR 51.2.2.4

~Ross

[RossD20Studios]

One more question for @ajwfrost and @marchbold -

Does Screen.mainScreen.safeArea and Display ANE detect cut-outs on desktop/laptop devices (for example, the cut-out on the MBP)?

[ajwfrost]

@RossD20Studios thanks for all that!

• For us, it would be most useful if you went for 'A' because then we'd know if that solves the issue. For your end users it's probably better to go with 'B' though...

• On macOS we hadn't actually got this supported, that's a good spot. We'll put that into our next release (sadly too late for 51.2.2.5 which is in progress..)

[marchbold]

@RossD20Studios Hi, yes the numbers reported in the insets are like margins from each edge, so if you are looking for a rectangular area then you'll need to combine this with the screen width.

@ajwfrost I'm interested to know why you are suggesting fullscreen to be true as the best approach? Could you elaborate on the difference on Android here? Historically we have always told our users to use false by default and then use our display modes (via the Application ANE) as this seemed to apply the "default" android view settings.

[marchbold]

Also @RossD20Studios I'd be interested in your experience on Samsung devices, these have been notorious for not following the guidelines from Google so always seem to have a quirk we need to account for.

[ajwfrost]

I'm interested to know why you are suggesting fullscreen to be true as the best approach?

@marchbold sorry, to clarify, if you have fullScreen set to 'true' then the safe area is likely to be accurate: if you have fullScreen set to 'false' then the content should naturally fit into the safe area on the device hence you can ignore the safeArea value. It's very possible that the safeArea rectangle is only correct when the app is full screen.

More generally though, I think Android are moving towards a 'full screen only' approach with Android 16? i.e. https://developer.android.com/about/versions/16/behavior-changes-16#adaptive-layouts "Apps also fill the entire display window, regardless of aspect ratio or a user's preferred orientation, and pillarboxing isn't used."

[RossD20Studios]

Hey @ajwfrost - I just released v2.76.0 yesterday which included the change you requested I test:

AIR 51.1.4 with the <displayCutoutMode>always</displayCutoutMode>

I haven't had a chance to follow-up with players who initially reported the bug. However, a new player emailed me a screenshot today explaining that after updating to v2.76.0, they are now experiencing the very same bug with the black bars that wasn't there before.

Here's their screenshot. I don't know yet what device this was on, but the frustrating concern is that changing to <displayCutoutMode>always</displayCutoutMode> seemed to cause the issue on devices where it wasn't present before.

I'll provide more info as I receive it from players. What steps would be ideal to test next? Should I try updating to AIR 51.2.5 or implement @marchbold 's ANE and give that a whirl?

~Ross

[ajwfrost]

Hi @RossD20Studios - that's a bit odd for it to suddenly start happening with that change. Can you confirm what your targetSdk version is? And if you can then find out what Android version is running there it could help, although I can't see how having the cut-out set to 'always' would end up with this black bar. Unless you are drawing any black area based on safe mode? The code logic you had pasted in earlier seems to add in a black scrim area but does this actually shift the content down? If the phone is the right way up I thought you didn't have the black area, you just shifted some of the important controls?

I think you would need to get some values from the phone i.e. to find out what the actual device values you're getting for the bounds and safe area, where the scrim is calculated at, etc. Currently I'm not 100% sure whether this is the OS giving you some padding based on app settings, or whether it's your code logic that's doing this maybe due to invalid inputs..

But for now - probably worth checking the behaviour using the ANE, and if that works we can then start comparing what we've got coming out of the built-in API...

thanks

[RossD20Studios]

Hi @ajwfrost -

The player who reported the recent UI breakage is using: DOOGEE S86 Android 10

They've been playing since Abalon v2.41 and only recently observed this error after I made the XML attribute change in v2.76. I've reached out to the other players that reported the screen cut-out and hoping they'll respond whether or not this resolved the cut-off issue for their devices.

Here's my Android target from the app descriptor XML: <uses-sdk android:minSdkVersion="24" android:targetSdkVersion="35" />

I will try the ANE for the next version. And I'll see about adding the device bounds and safe area information to my reporting.

Thanks,

~Ross

[RossD20Studios]

I received confirmation from our Samsung Galaxy S10 player today that changing <displayCutoutMode>always</displayCutoutMode> did not resolve the UI issue.

I'm working to release our next update soon with the ANE and I'll keep you both updated. Thanks!

[RossD20Studios]

More reports from users claiming that changing to <displayCutoutMode>always</displayCutoutMode> broke previous functionality:

Huawey P40

Also a few more (though I didn't get the device model - just that they said in their review of the game it was previously working, now doesn't).

I have implemented reporting for safeArea on Android. In addition to the display resolution, it will now show me what the safeArea is and which algorithm it used (either the ANE or the Native) depending on whether the ANE is supported. It'll also show the native safeArea if the ANE safeArea works and is different than the native safeArea so we can compare.

Is there anyway to get the device model? I would love to include this in the reports as well.

[marchbold]

If you are still using the Application ANE you can use a few properties on the device:

trace( Application.service.device.model );
trace( Application.service.device.product );
trace( Application.service.device.brand );
trace( Application.service.device.manufacturer );

https://docs.airnativeextensions.com/asdocs/application/com/distriqt/extension/application/Device.html

[RossD20Studios]

I'm not sure that I have access to Application ANE. I'm on the GameDev Distriqt subscription, but I'm not seeing it.

Also, I'm experiencing another odd problem. When I publish my app through Google Play and download it from the play store (internal dev build), the Display ANE is causing some kind of issue that prevents the game from running.

When I try to debug it to figure out what's happening, it runs just fine off the local build - without any other changes.

If I remove the Display ANE, it works fine...

Any idea what might cause this?

[marchbold]

Hmmm that seems strange, can you grab the logs?

[RossD20Studios]

Here is one of the errors I encountered:

ReferenceError: Error #1081
        at gameServices::DisplayService/get safeArea()
        at main::MainAndroid/getCustomSafeArea()
        at main::MainAbalon$/getSafeArea()
        at AA_Global$/get safeArea()
        at view.screenLayouts::BattleHUDScreenLayout/orient()
        at view::GameView/onOrient()
        at view::ViewModule/orient()
        at view::MainStarlingView/orient()
        at main::MainAbalon/changeOrientation()
        at main::MainAbalon/orientationChangeHandler()
        at com.d20studios.events::D20_EventDispatcher$/dispatch()
        at main::AppInfo/handleOrientationChange()

Here is the class in which the error occurred and where it occurred:

package gameServices{
	import com.distriqt.extension.display.Display;
	import com.distriqt.extension.display.insets.Insets;
	import flash.display.Screen;
	import flash.geom.Rectangle;

	public class DisplayService{
		private static var displayServiceSupported:Boolean;

		public function DisplayService(){
			try{
				if(Display.isSupported){
					displayServiceSupported = true;
				}
			}
			catch(e:Error){
			}
		}
		public function get safeArea():Rectangle{
			if(displayServiceSupported){
				var insets:Insets = Display.instance.windowInsets.getSafeArea();
				var safeArea:Rectangle = Screen.mainScreen.bounds;
				safeArea.x = insets.left; safeArea.width -= insets.left;
				safeArea.y = insets.top; safeArea.height -= insets.top;
				safeArea.width -= insets.right;
				safeArea.height -= insets.bottom;
			}
			else{
				safeArea = Screen.mainScreen.safeArea;
			}
			return safeArea;
		}
	}
}

To fix it, I changed: - if(displayServiceSupported){ + if(displayServiceSupported && Display.instance && Display.instance.windowInsets){

For the final class, I also added this method so I could determine in my bug reporting/session logs which version of the safe Area the application uses:

		public function get safeAreaType():String{
			if(displayServiceSupported && Display.instance && Display.instance.windowInsets){
				return "ANE";
			}
			return "Native";
		}

Once I made the changes, the application now loads and boots up without throwing an exception (at least one that my logs are currently able to catch, I suspect a bug is likely still happening before my logging takes place) - but it hangs at the login screen. This is very odd because it should be able to boot into the game whether an internet connection is available or not. Also, it should be restoring the cloud save (with login credentials) as soon as the Google Play user logs in.

I've attached the logcat file for the production version running through Google Play Store download on my internal testing track. The actual package name is air.com.d20studios.summonersfate

logfile.txt

[RossD20Studios]

AbalonGooglePlay-app.xml

Here's my complete AppDescriptor XML for reference.

One thought I had might be if any of the Google Play libraries using in the Display ANE might be different from my existing ANE versions, though I would have expected that library class conflict at compile time. But, maybe, when the libraries are run through Google Play Store (the app does appear to exhibit different behavior when its downloaded through the play store vs. debugger version) - maybe this is where a conflict is being introduced that causes the ANE to fail.

[RossD20Studios]

I figured it out... it was the obfuscation script (which only runs for release builds)! Sorry about that, my bad. While I had included the Display.swc in the ANE folders the script runs through, I didn't realize I still needed to explicitly add the SWC for exception.

Let me know how I can get access to the Application ANE so I can add the device models to the session reporting. That way, I can get you both an extensive list of Android devices and what safeAreas are being reported for them for both the Native AIR code and the Display ANE code.

Also, does the Application ANE require any special permissions to get device info? Ideal if I can avoid adding more permissions :)