Use baseline profiles to improve startup performance

[Mugurell]

(Spoiler: other apps saw even 30% - 40% startup improvements)

Baseline profiles

During installation, ART performs Ahead-of-time (AOT) compilation of methods in the profile, resulting in those methods executing faster. If the profile contains methods used in app launch or during frame rendering, the user experiences faster launch times and/or reduced jank.

 

From the cloud.

Android 9 (API level 28) introduced ART optimizing profiles in Play Cloud to improve app startup time and will usually be available the next day after a release.

Disadvantages:

• works only for Android 9+
• app updates clear the Cloud profiles data so with our fast release cycles users might not see big improvements

 

Manually created and shipped with every new release

Advantages:

• will work on Android 7+
• allows to manually tweak the profiles to best suit our needs
• data will continue to be aggregated in the cloud to improve this

Disadvantages:

• creating, tweaking and testing the improvements is a manual process for which Macrobenchmark should be used.

How baseline profiles work and how to create them locally - https://developer.android.com/topic/performance/baselineprofiles#creating-profile-rules

 

Success stories:

https://android-developers.googleblog.com/2022/01/improving-app-performance-with-baseline.html

cc @mcomella

┆Issue is synchronized with this Jira Task

[mcomella]

Thanks for sharing. As I understand the affected users:

• Users on Android 7 - 8.1 (API 24-27) will get performance benefits of ahead-of-time (AoT) compilation
• Users on Android 9 (API 28) and later will get extra performance benefits of AoT until about a day after an update is released (when the cloud profiles are ready)

To prioritize, it'd be helpful to know:

• How much work is implementing this? Since it doesn't need to run on a real device, it's maybe not too much (though coming up with a representative test may be challenging)
• What is the impact of enabling/disabling this functionality?
• What proportion of users are affected all of the time? What proportion of users are affected for a day after updates (note: this set of users will grow over time)?

Side note: our performance tests probably don't use these rules since we don't install our Nightly builds from Google Play.

[Amejia481]

I've been trying to integrate this into Fenix but it's been more challenging than I initially expected, as it requires a new alpha01 gradle tools update see.

Note: The minimum working version of com.android.tools.build:gradle is 7.1.0-alpha01, but we recommend using 7.3.0-beta01 or higher that comes with build improvements when using Baseline Profiles.

[FelixZhang00]

@Amejia481 I think baseline profiles already work in Firefox. Using this command adb shell cmd package dump-profiles org.mozilla.firefox, you can see the result in /data/misc/profman/org.mozilla.firefox-primary.prof.txt

=== Dex files  ===
base.apk [checksum=e242a1ed]
base.apk!classes2.dex [checksum=45cba755]
=== profile ===
ProfileInfo: empty
=== reference profile ===
ProfileInfo:
base.apk [index=0] [checksum=e242a1ed]
	hot methods: 
		void androidx.compose.animation.AndroidFlingSpline$FlingResult.<init>(float, float)[], 
		void androidx.compose.animation.AndroidFlingSpline.<clinit>()[], 
		void androidx.compose.animation.AndroidFlingSpline.<init>()[], 
		androidx.compose.animation.AndroidFlingSpline$FlingResult androidx.compose.animation.AndroidFlingSpline.flingPosition(float)[], 
		void androidx.compose.animation.FlingCalculator$FlingInfo.<init>(float, float, long)[], 
...

[Amejia481]

Awesome thank you so much for taking a look at it! I'll double check how we can integrate it into our build system.

[Mugurell]

@FelixZhang00 Just curious, have you tested this on an Android 9 or later device? - in which case you are probably seeing the cloud profiles automatically made available for all apps when running on Android P or later.

[FelixZhang00]

@Mugurell I tested this on Android 12

[Amejia481]

Thanks @Mugurell for the clarification, that could be the case @FelixZhang00 that what you are seeing it's a cloud profile provided by the Google Play. The goal of the ticket is to configure them as part of our build process to create our own profiles and being able to see locally if we are introducing regressions.

[mcomella]

While debugging https://github.com/mozilla-mobile/perf-frontend-issues/issues/236, I took cold main first frame measurements on a few different builds (caveat: only 8 replicates instead of the usual 25). For each build, I list comma-separated values: each one represents the median milliseconds of the test with 8 replicates.

• Nightly taskcluster: 540, 558.5, 720, 723ms (note: the wide gap here and below is due to https://github.com/mozilla-mobile/perf-frontend-issues/issues/236)
• Nightly Play Store: 565, 717
• Release: 549, 561

We'd expect to see baseline profiles on release (unless my device hadn't downloaded them yet or a new build was released less than 24hrs ago) but we might not see them on Nightly from the Play Store (since we update that build every 24hrs). However, ignoring the slow outliers on Nightly builds (which are likely an artifact of https://github.com/mozilla-mobile/perf-frontend-issues/issues/236), we don't see a significant performance difference between any of our builds. This implies that, for main first frame start up, baseline profiles aren't significant.

This could make sense though – we're only executing code that runs once (on start up) so it's unlikely to get optimized. The code that would be optimized and where we might see a difference – the UI code – hasn't really run yet.

[cpeterso]

Moved to bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1807350

Change performed by the Move to Bugzilla add-on.