Minimal Containers for Reproducible Builds testing for android app - WIP

[emanuelb]

This issue is about testing reproducible builds of android app via Minimal Containers, currently comparing compilation to mainnet file in v2 tag at: https://github.com/hsjoberg/blixt-wallet/releases/download/v0.2.0/blixt-wallet-mainnet-0.2.0.apk

remaining tasks:

1. Run jadx/apktool/diffoscope to see the detailed diff
2. Decompile the Hermes bytecode of index.android.bundle file to see whats the diff in it.
3. Create Containerfile for creating the .aar file! and also test it for reproducibility! (compare to file in assets in v2 release)
4. Create a apk from bundle instead? (There is BNDLTOOL* files in apk from github)

Results on running diff --recursive --brief ./app-chainmainnet-normal-release-v2-t2/ ./blixt-wallet-mainnet-0.2.0/

Files ./app-chainmainnet-normal-release-v2-t2/AndroidManifest.xml and ./blixt-wallet-mainnet-0.2.0/AndroidManifest.xml differ

Files ./app-chainmainnet-normal-release-v2-t2/assets/index.android.bundle and ./blixt-wallet-mainnet-0.2.0/assets/index.android.bundle differ

Files ./app-chainmainnet-normal-release-v2-t2/classes2.dex and ./blixt-wallet-mainnet-0.2.0/classes2.dex differ

Files ./app-chainmainnet-normal-release-v2-t2/classes.dex and ./blixt-wallet-mainnet-0.2.0/classes.dex differ

Only in ./blixt-wallet-mainnet-0.2.0/META-INF: BNDLTOOL.RSA
Only in ./blixt-wallet-mainnet-0.2.0/META-INF: BNDLTOOL.SF

Only in ./app-chainmainnet-normal-release-v2-t2/META-INF: CERT.RSA

Only in ./app-chainmainnet-normal-release-v2-t2/META-INF: CERT.SF

Files ./app-chainmainnet-normal-release-v2-t2/META-INF/MANIFEST.MF and ./blixt-wallet-mainnet-0.2.0/META-INF/MANIFEST.MF differ

Only in ./blixt-wallet-mainnet-0.2.0/res/xml: splits0.xml

Files ./app-chainmainnet-normal-release-v2-t2/resources.arsc and ./blixt-wallet-mainnet-0.2.0/resources.arsc differ

notes:

• ndkVersion downloaded matched to what declared in ./app/build.gradle file (to pass NDK not found error)

stuff to look for later regarding minimizing deps installed:

• binutils & binutils-arm-none-eabi installed for strip & objcopy tools, might not be needed.
• python3 make g++ are installed to pass yarn install command, might be not needed if --ignore-optional is used?
• avoid the npm dependency, report issue upstream if it's still the case when all the packages are upgraded to latest version and building again without installing npm fail, in previous compiling the yarn gen-proto; step require npm installation (upstream may fix it by working as well if yarn installed instead)

ContainerFile2Alpine file content:

FROM frolvlad/alpine-glibc

RUN set -ex; \
    apk update; \
    apk add --no-cache \
        git \
        npm \
        yarn \
        make \
        g++ \
        python3 \
        binutils \
        binutils-arm-none-eabi \
        openjdk8; \
    adduser -D appuser;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="production"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/blixt/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/blixt/; \
    git clone https://github.com/hsjoberg/blixt-wallet; \
    cd ./blixt-wallet/; \
    git checkout v0.2.0; \
    cd ./android/lndmobile; \
    wget https://github.com/hsjoberg/blixt-wallet/releases/download/v0.2.0/Lndmobile.aar; \
    echo "b7382f852842b790a2f40439213b17fa000b79cd1eab8dac7e9b436154b23131  Lndmobile.aar" | sha256sum -c; \
    cd /home/appuser/app/blixt/blixt-wallet/; \
    git submodule update --init; \    
    yarn install --production; \
    yarn build-tor-lib; \
    # require npm package installed?
    yarn gen-proto; \
    cd /home/appuser/app/blixt/blixt-wallet; \
    # it's in devDep instead of deps
    yarn add metro-react-native-babel-preset remote-redux-devtools @babel/runtime; 

WORKDIR /home/appuser/app/blixt/blixt-wallet/android/

Then run:

podman build --rm -t blixt2_build_apk -f ContainerFile2Alpine
podman run --rm --name blixt2_built_apk -ti blixt2_build_apk

inside the container run: (first command will fail due to missing ndk it's ok, second command will install it)

./gradlew assembleChainmainnetNormal
/home/appuser/app/sdk/tools/bin/sdkmanager --sdk_root=/home/appuser/app/sdk/ --install "ndk;20.0.5594570"
./gradlew assembleChainmainnetNormal

[hsjoberg]

@emanuelb Thank you for all your work on this.

Create a apk from bundle instead? (There is BNDLTOOL* files in apk from github)

FYI, I use app bundles when I publish it to google play. I extracted a "universal" apk using bundletool to get the apk used in the 0.2.0 release.

[emanuelb]

.aar generation tracked in issue: https://github.com/hsjoberg/blixt-wallet/issues/319 will update the result of diffing later when managed to compile it.

I extracted a "universal" apk using bundletool to get the apk used in the 0.2.0 release.

I assume you did the same for 0.2.1 release, and you didn't use the Containerfile in this issue for it, right? (it's for testing purposes) Anyway by doing the same process with bundle tool on v0.2.1 the result is more reproducible output (only diff in classes.dex & classes2.dex matters):

diff --recursive --brief ./app-chainmainnet-normal-release-21bu/ ./blixt-wallet-mainnet-0.2.1/
Files ./app-chainmainnet-normal-release-21bu/classes2.dex and ./blixt-wallet-mainnet-0.2.1/classes2.dex differ
Files ./app-chainmainnet-normal-release-21bu/classes.dex and ./blixt-wallet-mainnet-0.2.1/classes.dex differ
Only in ./blixt-wallet-mainnet-0.2.1/META-INF: BNDLTOOL.RSA
Only in ./blixt-wallet-mainnet-0.2.1/META-INF: BNDLTOOL.SF
Files ./app-chainmainnet-normal-release-21bu/META-INF/MANIFEST.MF and ./blixt-wallet-mainnet-0.2.1/META-INF/MANIFEST.MF differ

from apktool/jadx diffs:

not sure, maybe java version different? what java version installed where the android compilation took place? I tried both java8 & java11 but there still diff... maybe dep locking missing somewhere?

-- /tmp/1aaa/smali/com/aakashns/reactnativedialogs/modules/DialogAndroid$k.smali
+++ /tmp/1bbb/smali/com/aakashns/reactnativedialogs/modules/DialogAndroid$k.smali
@@ -26,6 +26,11 @@
 # direct methods
 .method constructor <init>(Lcom/aakashns/reactnativedialogs/modules/DialogAndroid;Lcom/facebook/react/bridge/Callback;)V
     .locals 0
+    .annotation system Ldalvik/annotation/Signature;
+        value = {
+            "()V"
+        }
+    .end annotation

ordering issues, maybe can be overcomed by disorderfs tool:

--- /tmp/1aaa/smali/co/apptailor/googlesignin/RNGoogleSigninModule.smali
+++ /tmp/1bbb/smali/co/apptailor/googlesignin/RNGoogleSigninModule.smali
@@ -6,9 +6,9 @@
 # annotations
 .annotation system Ldalvik/annotation/MemberClasses;
     value = {
-        Lco/apptailor/googlesignin/RNGoogleSigninModule$g;,
+        Lco/apptailor/googlesignin/RNGoogleSigninModule$f;,
         Lco/apptailor/googlesignin/RNGoogleSigninModule$e;,
-        Lco/apptailor/googlesignin/RNGoogleSigninModule$f;
+        Lco/apptailor/googlesignin/RNGoogleSigninModule$g;
     }
 .end annotation

and more similar differences exists...

The npm & uglify-js installation in below script can be avoided after fixing issue https://github.com/hsjoberg/blixt-wallet/issues/320

ContainerFile21Alpine content:

FROM frolvlad/alpine-glibc

RUN set -ex; \
    apk update; \
    apk add --no-cache \
        git \
        npm \
        yarn \
        openjdk11; \
    adduser -D appuser;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="production"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/blixt/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/sdk/; \
    wget https://github.com/google/bundletool/releases/download/1.5.0/bundletool-all-1.5.0.jar; \
    echo "b7452e243a8bb32762ef74017f68291c685be0b3006b4b199fb94a7e7793dc85  bundletool-all-1.5.0.jar" | sha256sum -c; \
    cd /home/appuser/app/blixt/; \
    git clone https://github.com/hsjoberg/blixt-wallet; \
    cd ./blixt-wallet/; \
    git checkout v0.2.1; \
    cd ./android/lndmobile; \
    wget https://github.com/hsjoberg/blixt-wallet/releases/download/v0.2.1/Lndmobile.aar; \
    echo "cf51ec8d63ec4c4370aca273b1f1d5cdcf075e59db65f16504cd798c875df13c  Lndmobile.aar" | sha256sum -c; \
    cd /home/appuser/app/blixt/blixt-wallet/; \
    git submodule update --init; \    
    yarn install --frozen-lockfile --ignore-optional --ignore-scripts; \
    yarn build-tor-lib;

WORKDIR /home/appuser/app/blixt/blixt-wallet/android/

Then run:

podman build --rm -t blixt21_build_apk -f ContainerFile21Alpine
podman run --rm --name blixt21_built_apk -ti blixt21_build_apk

inside the container run: (first gradlew command will fail due to missing ndk it's ok, next command after it will install it)

cd /home/appuser/app/blixt/blixt-wallet;
yarn add uglify-js;
yarn gen-proto;
cd /home/appuser/app/blixt/blixt-wallet/android/;
./gradlew bundleChainmainnet
cd /home/appuser/app/sdk/;
mkdir ndk;
wget https://dl.google.com/android/repository/android-ndk-r22b-linux-x86_64.zip
echo "ac3a0421e76f71dd330d0cd55f9d99b9ac864c4c034fc67e0d671d022d4e806b  android-ndk-r22b-linux-x86_64.zip" | sha256sum -c;
unzip android-ndk-r22b-linux-x86_64.zip
rm android-ndk-r22b-linux-x86_64.zip
mv android-ndk-r22b /home/appuser/app/sdk/ndk/22.1.7171670/
./gradlew bundleChainmainnet
cd /home/appuser/app/blixt/;
java -jar /home/appuser/app/sdk/bundletool-all-1.5.0.jar build-apks --bundle=/home/appuser/app/blixt/blixt-wallet/android/app/build/outputs/bundle/chainmainnetNormalRelease/app-chainmainnet-normal-release.aab --mode=universal --output=/home/appuser/app/blixt/blixt-chainmainnet-normal-release-appbundle.apks
unzip blixt-chainmainnet-normal-release-appbundle.apks

Then copy the universal.apk outside container:

podman cp blixt21_built_apk:/home/appuser/app/blixt/universal.apk ~/blixt-app-chainmainnet-normal-release-21.apk

also opened issue in diffoscope which include details on how to decompile/disassemble hermes files: (There was diff in previous version 0.2.0) Support "Hermes JavaScript bytecode" files (React-Native)" https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/248

[Giszmo]

Is this issue still up to date? I just updated the wallet's review at WalletScrutiny to "not reproducible" but I guess it's close to being reproducible? App Bundles might be an additional challenge as none of our reproducible apps uses App Bundles so far but @emanuelb also did some work on how to achieve this in general.

[hsjoberg]

@Giszmo no progress on this one just yet unfortunately. I hope to eventually have time looking into making Blixt versions reproducible.

[emanuelb]

Building with debian based image latest version v0.5.1 with Containerfile

FROM docker.io/debian:sid-slim

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        npm \
        git \
        yarnpkg \
        wget \
        unzip \
        openjdk-11-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser;
    
USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="development"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/blixt/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/sdk/; \
    wget https://github.com/google/bundletool/releases/download/1.9.0/bundletool-all-1.9.0.jar; \
    cd /home/appuser/app/blixt/; \
    git clone https://github.com/hsjoberg/blixt-wallet; \
    cd ./blixt-wallet/; \
    git checkout v0.5.1; \
    cd ./android/lndmobile; \
    wget https://github.com/hsjoberg/blixt-wallet/releases/download/v0.5.1/Lndmobile.aar; \
    cd /home/appuser/app/blixt/blixt-wallet/; \
    git submodule update --init; \    
    yarnpkg install --frozen-lockfile --ignore-optional --ignore-scripts; \
    yarnpkg build-tor-lib;

WORKDIR /home/appuser/app/blixt/blixt-wallet/android/

Then run:

podman build --pull --rm -t blixt_ver501_build_apk -f ContainerFile5
podman run --rm --name blixt_ver501_build_apk -ti blixt_ver501_build_apk

inside the container run:

cd /home/appuser/app/blixt/blixt-wallet;
yarnpkg add uglify-js espree react-native-svg;
yarnpkg gen-proto;
cd /home/appuser/app/blixt/blixt-wallet/android/;
./gradlew bundleChainmainnet;
cd /home/appuser/app/blixt/;
java -jar /home/appuser/app/sdk/bundletool-all-1.9.0.jar build-apks --bundle=/home/appuser/app/blixt/blixt-wallet/android/app/build/outputs/bundle/chainmainnetNormalRelease/app-chainmainnet-normal-release.aab --mode=universal --output=/home/appuser/app/blixt/blixt-chainmainnet-normal-release-appbundle.apks;
unzip blixt-chainmainnet-normal-release-appbundle.apks;

compared to apk from: https://github.com/hsjoberg/blixt-wallet/releases/download/v0.5.1/blixt-wallet-mainnet-0.5.1.apk

Files ./Build/assets/index.android.bundle and ./Github/assets/index.android.bundle differ
Files ./Build/classes.dex and ./Github/classes.dex differ
Files ./Build/classes2.dex and ./Github/classes2.dex differ
Files ./Build/lib/arm64-v8a/libconceal.so and ./Github/lib/arm64-v8a/libconceal.so differ
Files ./Build/lib/arm64-v8a/libgojni.so and ./Github/lib/arm64-v8a/libgojni.so differ
Files ./Build/lib/arm64-v8a/libtor.so and ./Github/lib/arm64-v8a/libtor.so differ
Files ./Build/lib/armeabi-v7a/libconceal.so and ./Github/lib/armeabi-v7a/libconceal.so differ
Files ./Build/lib/armeabi-v7a/libgojni.so and ./Github/lib/armeabi-v7a/libgojni.so differ
Files ./Build/lib/armeabi-v7a/libtor.so and ./Github/lib/armeabi-v7a/libtor.so differ
Files ./Build/lib/x86/libconceal.so and ./Github/lib/x86/libconceal.so differ
Files ./Build/lib/x86/libgojni.so and ./Github/lib/x86/libgojni.so differ
Files ./Build/lib/x86/libtor.so and ./Github/lib/x86/libtor.so differ
Files ./Build/lib/x86_64/libconceal.so and ./Github/lib/x86_64/libconceal.so differ
Files ./Build/lib/x86_64/libgojni.so and ./Github/lib/x86_64/libgojni.so differ
Files ./Build/lib/x86_64/libtor.so and ./Github/lib/x86_64/libtor.so differ