Reproducible Build: Fails to build due to MapBox requirement

[xrviv]

Greetings team Mixin! 🙂

Danny here from walletscrutiny.com. We verify the reproducibility of Android apps. So far, we've reviewed over 6500+ Android apps and Bitcoin hardware wallets.

Describe the bug

• Build Failure
• Insufficient build instructions
• Non-verifiability

To Reproduce Steps to reproduce the behavior: We attempted to build again with the knowledge that an MR has been merged regarding Emanuel's issue. We assume that the app could be built from hereon.

We begin our manual attempt, with this Dockerfile:

FROM ubuntu:22.04

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    unzip \
    git \
    openjdk-17-jdk \
    && rm -rf /var/lib/apt/lists/*

ENV ANDROID_SDK_ROOT=/opt/android-sdk
ENV PATH=$PATH:$ANDROID_SDK_ROOT/cmdline-tools/latest/bin:$ANDROID_SDK_ROOT/platform-tools

ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
ENV PATH=$PATH:$JAVA_HOME/bin


RUN mkdir -p $ANDROID_SDK_ROOT/cmdline-tools
RUN cd $ANDROID_SDK_ROOT/cmdline-tools && \
    curl -L --retry 5 --retry-connrefused --insecure https://dl.google.com/android/repository/commandlinetools-linux-10406996_latest.zip -o commandlinetools.zip && \
    unzip commandlinetools.zip && \
    rm commandlinetools.zip && \
    mv cmdline-tools latest

RUN yes | sdkmanager --licenses
RUN sdkmanager "platform-tools" "platforms;android-35" "build-tools;35.0.0"

RUN sdkmanager "ndk;27.0.12077973"

RUN /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" \
    && echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /root/.bashrc \
    && eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"

We build the docker image:

cd /tmp/manualtest_one.mixin.messenger
docker build -t mixin-image . && docker run -it --rm -v "$(pwd)":/mnt mixin-image

From there, we execute:

# cd /mnt
# git clone https://github.com/MixinNetwork/android-app.git
# cd android-app
# export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
# export ANDROID_SDK_ROOT=/opt/android-sdk
# export PATH=$PATH:$ANDROID_SDK_ROOT/cmdline-tools/latest/bin:$ANDROID_SDK_ROOT/platform-tools:$JAVA_HOME/bin
# yes | sdkmanager --licenses
# sdkmanager "platforms;android-31" "build-tools;31.0.0"
# ./gradlew assembleRelease --stacktrace

The complete nosbin paste

After 3 attempts, we noticed a recurring problem whenever we tried to execute ./gradlew.

Could not resolve com.mapbox.maps:android:10.10.0.
     Required by:
         project :app

> Could not get resource 'https://maven.pkg.github.com/checkout/checkout-3ds-sdk-android/com/mapbox/plugin/maps-lifecycle/10.10.0/maps-lifecycle-10.10.0.pom'.
            > Username must not be null!

Expected behavior

1. The app builds successfully
2. We could then run a diff between the built vs the official artifact

Screenshots Screenshot of mapbox requirements posted on x

Desktop (please complete the following information):

• OS: Ubuntu 22.04
• Browser: Firefox
• Version: 130.01

Smartphone (please complete the following information):

• Device: n/a we used a build server
• OS: n/a we used a build server
• Browser n/a we used a build server
• Version n/a we used a build server

Additional context We used a debian build server + docker

[crossle]

Please wait a sec, we'll try fix this without download key.

[cedricfung]

@xrviv Now we removed mapbox, and the reproducible build script should work as expected, could you please try it again?

[xrviv]

Hi, will do this first thing in the morning.

[xrviv]

I attempted to build this several times today:

• build001

• build002

• build003

• Text file of build003

In one way or another, I encountered this problem:

Configuration 'releaseProtobuf' was resolved during configuration time.
This is a build performance and scalability issue.
See https://github.com/gradle/gradle/issues/2298
Run with --info for a stacktrace.
> Task :app:processReleaseGoogleServices FAILED

[Incubating] Problems report is available at: file:///home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/build/reports/problems/problems-report.html

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:processReleaseGoogleServices'.
> File google-services.json is missing. 
  The Google Services Plugin cannot function without it. 
  Searched locations: /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/Release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/google-services.json

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.12.1/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD FAILED in 1m 8s
12 actionable tasks: 12 executed
Configuration cache entry stored.

The build failed during the :app:processReleaseGoogleServices task because the required file google-services.json was not found in any of the expected locations. This file is essential for the Google Services Gradle plugin to configure Firebase (or other Google) services used by the app.

I will try to put in a "dummy" google-services.json file:

{
  "project_info": {
    "project_number": "1234567890",
    "project_id": "dummy-project-id",
    "firebase_url": "https://dummy-project-id.firebaseio.com",
    "storage_bucket": "dummy-project-id.appspot.com"
  },
  "client": [
    {
      "client_info": {
        "mobilesdk_app_id": "1:1234567890:android:dummyappid",
        "android_client_info": {
          "package_name": "one.mixin.messenger"
        }
      },
      "oauth_client": [],
      "api_key": [
        {
          "current_key": "DUMMY_API_KEY"
        }
      ],
      "services": {
        "analytics_service": {
          "status": "0"
        },
        "appinvite_service": {
          "status": "0",
          "other_platform_oauth_client": []
        },
        "ads_service": {
          "status": "0"
        }
      }
    }
  ],
  "configuration_version": "1"
}

With the dummy google-services.json file:

FAILURE: Build failed with an exception.

* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

[crossle]

Please try again.

[cedricfung]

Please try again.

This should be added to CI to ensure everything always works

[xrviv]

After a few days, I am attempting this again.

one.mixin.messenger

This is a build performance and scalability issue.
See https://github.com/gradle/gradle/issues/2298
Run with --info for a stacktrace.
> Task :app:processReleaseGoogleServices FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:processReleaseGoogleServices'.
> File google-services.json is missing. 
  The Google Services Plugin cannot function without it. 
  Searched locations: /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/src/Release/google-services.json, /home/danny/work/builds/android/one.mixin.messenger/2.3.1/android-app/app/google-services.json

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.10.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD FAILED in 2m 2s
11 actionable tasks: 11 executed
Configuration cache entry stored.

So, the same problem - I would have to create a dummy google-services.json file.

[cedricfung]

Maybe try the docker build command from this script https://github.com/MixinNetwork/android-app/blob/master/verify-mixin-apk.sh

Or you could verify the reproducibility with the script directly.

[xrviv]

Will attempt that after I try this approach with creating a dummy google-services.json file

{
  "project_info": {
    "project_number": "123456789012",
    "firebase_url": "https://one-mixin-messenger.firebaseio.com",
    "project_id": "one-mixin-messenger",
    "storage_bucket": "one-mixin-messenger.appspot.com"
  },
  "client": [
    {
      "client_info": {
        "mobilesdk_app_id": "1:123456789012:android:abcdef123456",
        "android_client_info": {
          "package_name": "one.mixin.messenger"
        }
      },
      "oauth_client": [],
      "api_key": [
        {
          "current_key": "AIzaSyDummyKeyForMixinMessenger"
        }
      ],
      "services": {
        "analytics_service": {
          "status": 2,
          "analytics_property": {
            "tracking_id": "UA-12345678-1"
          }
        },
        "cloud_messaging": {
          "status": 2,
          "apns_config": []
        }
      }
    }
  ],
  "configuration_version": "1"
}

The build with the dummy google-services.json seems to have spawned a different problem.

WARNING: R8: The method `com.twilio.video.VideoCodec com.sumsub.sns.internal.features.presentation.videoident.chat.SNSVideoChatController.a(com.twilio.video.VideoCodec)` does not type check and will be assumed to be unreachable.

> Task :app:uploadSentryProguardMappingsRelease
> compressing mappings
> uploading mappings
error: Auth token is required for this request. Please run `sentry-cli login` and try again!

Add --log-level=[info|debug] or export SENTRY_LOG_LEVEL=[info|debug] to see more output.
Please attach the full debug log to all bug reports.


> Task :app:uploadSentryProguardMappingsRelease FAILED
ASM Instrumentation process wasn't able to resolve some classes, this means that
the instrumented classes might contain corrupt stack frames. Make sure the
dependencies that contain these classes are on the runtime or the provided
classpath. Otherwise, the jvm might fail to load the corrupt classes at runtime
when running in a jvm environment like unit tests.

Classes that weren't resolved:
> org.iq80.leveldb.DBIterator
> org.iq80.leveldb.DBException
> androidx.camera.extensions.impl.advanced.ImageReaderOutputConfigImpl
> androidx.camera.extensions.impl.advanced.MultiResolutionImageReaderOutputConfigImpl
> com.google.mlkit.vision.face.FaceDetector
> com.google.mlkit.vision.common.InputImage
> com.google.common.util.concurrent.ListenableFuture
> org.bouncycastle.jce.spec.ECNamedCurveParameterSpec
> org.bouncycastle.math.ec.ECPoint
> org.bouncycastle.math.ec.ECCurve
-- 60 more classes --


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:uploadSentryProguardMappingsRelease'.
> An error occurred while executing task 'uploadSentryProguardMappingsRelease'. Please check the detailed sentry-cli output above.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 18m 17s
71 actionable tasks: 62 executed, 9 up-to-date
Configuration cache entry reused.

I will now attempt as you suggested above.

[xrviv]

I modified the verify-mixin-apk.sh to skip the adb processing portion (since I am doing this on a remote debian server).

#!/bin/bash

set -e

cd "$(git rev-parse --show-toplevel)"

echo "Skipping ADB processing step (APK extraction from device)."

echo "Building mixin APK from source code. This might take a while (20-30 minutes)..."

mkdir -p ./output-apk
docker run --rm \
  -v "$(pwd)":/project \
  -v "$(pwd)/output-apk":/home/gradle/app/build/outputs/apk/release \
  mingc/android-build-box bash -c 'cd /project; ./gradlew :app:assembleRelease'

if [ -f "./output-apk/app-release-unsigned.apk" ]; then
    echo "Build succeeded! The built APK is located at ./output-apk/app-release-unsigned.apk"
else
    echo "Build failed: APK not found."
    exit 1
fi

2025-03-12.one.mixin.messenger_v2.3.1-build002.txt

Build failed

Note: Recompile with -Xlint:unchecked for details.
com.uber.autodispose.lint.AutoDisposeIssueRegistry in /root/.gradle/caches/8.10.2/transforms/0b827862c272120f735564d3f424dc2c/transformed/autodispose-android-1.4.1/jars/lint.jar does not specify a vendor; see IssueRegistry#vendor----- End of the daemon log -----


FAILURE: Build failed with an exception.

* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

I will retry again, with one modification ./gradlew bundleRelease to generate the AAB.

[xrviv]

Modified your script, with:

#!/bin/bash
# verify-mixin-aab.sh
#
# This script builds an AAB (Android App Bundle) for Mixin Messenger from source.
#
# Note: A known issue with autodispose-android 1.4.1 causes lint to crash because its lint.jar
# does not specify a vendor. To work around this, we disable lint checks using "-x lint".
#
# The built AAB will be output to the ./output-aab directory.
#
set -e

cd "$(git rev-parse --show-toplevel)"

echo "Building mixin AAB from source code. This might take a while (20-30 minutes)..."

mkdir -p ./output-aab
docker run --rm \
  -v "$(pwd)":/project \
  -v "$(pwd)/output-aab":/home/gradle/app/build/outputs/bundle/release \
  mingc/android-build-box bash -c 'cd /project; ./gradlew :app:bundleRelease -x lint'

if [ -f "./output-aab/app-release.aab" ]; then
    echo "Build succeeded! The built AAB is located at ./output-aab/app-release.aab"
else
    echo "Build failed: AAB not found."
    exit 1
fi

currently testing both remotely and locally.

Remotely, this is the result:

2025-03-12.one.mixin.messenger_v2.3.1-build003.txt

[xrviv]

Manual build fails as well

build004

[crossle]

Manual build fails as well

build004

Try master code please.

[xrviv]

build005 - master

I have to rest, been doing this for most of the day.