SimpMusic icon indicating copy to clipboard operation
SimpMusic copied to clipboard

Reproducible Build failed since v0.2.18-hf

Open IzzySoft opened this issue 5 months ago • 26 comments

… due to differences in M7.json (aboutLibraries):

  -rw----     0.0 fat      612 b-      304 defN 1981-01-01 01:01:02 3515f2bb res/M2.xml
- -rw----     0.0 fat   478139 b-    42878 defN 1981-01-01 01:01:02 98fac9dc res/M7.json
+ -rw----     0.0 fat   478259 b-    43061 defN 1981-01-01 01:01:02 83bf4207 res/M7.json
  -rw----     0.0 fat     1196 b-      397 defN 1981-01-01 01:01:02 0db276ad res/M7.xml

The JSON itself:

         "CC0-1.0": {
-            "content": "Creative Commons CC0 1.0 Universal\n\n<<beginOptional;name=ccOptionalIntro>> CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN ATT
+            "content": "Creative Commons Legal Code\n\nCC0 1.0 Universal\n\n    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE\n    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN\n    ATTORNEY-CLIEN
             "hash": "CC0-1.0",

Any clues what changed there in the last 18 hours that maybe wasn't pinned – or was it something else? It's just CC0 1.0 Universal\n\n<<beginOptional;name=ccOptionalIntro>> vs. Legal Code\n\nCC0 1.0 Universal\n\n , that seems to be the only difference.

IzzySoft avatar Jul 23 '25 20:07 IzzySoft

OK, have to mark that version "RB failed" now after the 24h limit. Hopefully we get this fixed with the next release then.

IzzySoft avatar Jul 24 '25 20:07 IzzySoft

Hmmm, I don't know why, do you have any idea @IzzySoft? In reproducible builds, too hard to know why

maxrave-dev avatar Jul 30 '25 09:07 maxrave-dev

I don't know what exactly caused this, so I can only guess. And the first thing coming to mind there is: have you built the APK from a clean tree at the commit the tag points to – or were there local changes, or artifacts remaining from prior builds? Those 2 are the most likely causes. But they should not happen when building via CI – like here, with Github actions, as then there's always a fresh copy checked out into a "clean environment", and built from that. The release APK was from that workflow, wasn't it?

IzzySoft avatar Jul 30 '25 11:07 IzzySoft

Yes, the APKs from this build (https://github.com/maxrave-dev/SimpMusic/actions/workflows/android-release.yml), and I create the release manually

maxrave-dev avatar Aug 01 '25 03:08 maxrave-dev

Well… We cannot use your build script here, as it enforces signing (and aborts before the build if no signing keys/credentials have been passed). But the steps we use are similar, except for the signing of course:

- sed -r '/abi \{/,/}/ { s/include\(.*\)/include("arm64-v8a")/ ; s/isUniversalApk = true/isUniversalApk = false/ }' -i app/build.gradle.kts
- chmod +x gradlew
- ./gradlew assembleFossRelease

(the initial sed is just to not waste resources: we can only check one APK). ./gradlew clean is not needed, as we use ephemeral containers to build (as Github actions do as well AFAIK) – so no local changes or left-over artifacts to care about, though it of course cannot hurt. The only thing I can think about is some "unpinned" dependency, which was updated between your build and ours. But I'm not familiar enough with Kotlin to know how to verify that…

IzzySoft avatar Aug 01 '25 10:08 IzzySoft

v0.2.19 failed as well, same M7.json stuff.

IzzySoft avatar Aug 11 '25 19:08 IzzySoft

@IzzySoft btw, I still see the 0.2.18-hf still deployed in F Droid, Is your system normally?

maxrave-dev avatar Aug 12 '25 15:08 maxrave-dev

I cannot tell about what is deployed at F-Droid, so I need to guess what your question means. At IzzyOnDroid, RB runs on a separate track – so a failed RB doesn't stop your update from being published, it just keeps the green shield from coming up. In the future, our clients (Droid-ify and Neo Store) plan to have an option to only install/update with the shield up. But we'll leave it in the hands of those using the apps to decide what level of confirmation they want/need. A failed RB does not necessarily mean "red alert: Klingons on the starship bridge!" It can be something as simple as… an M7.json being messed up. That's plain text, nothing critical at all.

IzzySoft avatar Aug 12 '25 16:08 IzzySoft

So I ask the author of AboutLibraries but he had no idea about this error, so I need time to investigate, sorry for late

maxrave-dev avatar Aug 14 '25 01:08 maxrave-dev

Thank you for investigating! Yeah, it's nasty if one cannot pin-point things. Here at least we know which library it belongs to – other times we're sitting entirely in the dark… Best look with your investigations 🤞 Hopefully it's not too complicated, and easy to fix…

IzzySoft avatar Aug 14 '25 08:08 IzzySoft

v0.2.20 didn't even start building, looks like some error in your build.gradle:

Checking task: assembleFossRelease
Is full build: false
  
> Configure project :app
w: file:///build/repo/app/build.gradle.kts:28:9: Variable 'abis' is never used
Full build detected, enabling Sentry Auth Token
Failed to load SENTRY_AUTH_TOKEN from local.properties: /build/repo/local.properties (No such file or directory)
  
FAILURE: Build failed with an exception.

Note: "is full build: false", but then "full build detected, enabling sentry". Doesn't match. And strange, as there are only 2 occurrences of isFullBuild in app/build.gradle: one reading it from rootProject.extra, and one evaluating it for sentry. Even if I explicitly set it to false, using sed 's/val isFullBuild.*/val isFullBuild:\ Boolean = false/', the very same block kicks in, claiming "full build detected". Fully strange: same if I do sed 's/if \(isFullBuild\)/if (false)/' – and at that point I'm massively confused.

        if (false) {
            try {
                println("Full build detected, enabling Sentry DSN")
                val properties = Properties()
                properties.load(rootProject.file("local.properties").inputStream())
                buildConfigField(
                    "String",
                    "SENTRY_DSN",
                    "\"${properties.getProperty("SENTRY_DSN") ?: ""}\"",
                )
            } catch (e: Exception) {
                println("Failed to load SENTRY_DSN from local.properties: ${e.message}")
            }
        }

that block should never be executed – but maybe I do not understand Kotlin well enough there. Have to give up at this point, sorry. But that raises the question if your foss build now comes with Sentry enabled, too? In your case, that property file was most likely present.

IzzySoft avatar Aug 31 '25 19:08 IzzySoft

what gradle task you do to build @IzzySoft? if you use app:assembleFossRelease, the code block never execute, so can not be error

maxrave-dev avatar Sep 01 '25 09:09 maxrave-dev

Same. Until now, we just used assembleFossRelease. Tried with :app:assembleFossResult, but same result:

        build:
          - sed -r '/abi \{/,/}/ { s/include\(.*\)/include("arm64-v8a")/ ; s/isUniversalApk = true/isUniversalApk = false/ }' -i app/build.gradle.kts
          - chmod +x gradlew
          - ./gradlew :app:assembleFossRelease

If you wonder about the first line: to avoid resource waste, we try to build only the ABI we check, as we can only check one APK per run. Now comes the fun fact: if I remove that line, build proceeds, despite of throwing the same error:

> Configure project :
Checking task: :app:assembleFossRelease
Is full build: false

> Configure project :app
Full build detected, enabling Sentry Auth Token
Failed to load SENTRY_AUTH_TOKEN from local.properties: /build/repo/local.properties (No such file or directory)
Checking the license for package Android SDK Build-Tools 35 in /opt/sdk/licenses
License for package Android SDK Build-Tools 35 accepted.
Preparing "Install Android SDK Build-Tools 35 v.35.0.0".

But as soon as I even only disable the universal build, it fails. Would there be a gradle parameter to pass so only the specified ABI is being built?

And though the build succeeds if I leave out that line, it's not RB:

  -rw-r--r--  0.0 unx     8016 b-     4088 defN 1981-01-01 01:01:02 a084dec5 classes2.dex
- -rw-r--r--  0.0 unx  9279172 b-  3540429 defN 1981-01-01 01:01:02 528deb21 classes3.dex
+ -rw-r--r--  0.0 unx  9279172 b-  3540434 defN 1981-01-01 01:01:02 21f3d6b1 classes3.dex
  -rw-r--r--  0.0 unx  2597744 b-  1154105 defN 1981-01-01 01:01:02 d9ca069a classes4.dex
  -rw-r--r--  0.0 unx    10096 b-     4386 defN 1981-01-01 01:01:02 9734baa0 lib/arm64-v8a/libandroidx.graphics.path.so
  -rw-r--r--  0.0 unx 10151888 b-  5647956 defN 1981-01-01 01:01:02 44adf6e3 lib/arm64-v8a/libavcodec.so
@@ -626,7 +626,7 @@
  -rw----     0.0 fat      592 b-      317 defN 1981-01-01 01:01:02 4ee58e62 res/Lt.xml
  -rw----     0.0 fat      132 b-       75 defN 1981-01-01 01:01:02 214428f8 res/Lv.xml
  -rw----     0.0 fat      612 b-      304 defN 1981-01-01 01:01:02 fd002b0a res/M2.xml
- -rw----     0.0 fat   441080 b-    31694 defN 1981-01-01 01:01:02 64de51ae res/M7.json
+ -rw----     0.0 fat   442621 b-    32090 defN 1981-01-01 01:01:02 87598d61 res/M7.json
  -rw----     0.0 fat     1196 b-      397 defN 1981-01-01 01:01:02 0db276ad res/M7.xml

M7.json is aboutLibraries. Congrats, we had that before, something with API limits or so… And the diff of classes3.dex:

 |: move-result-object v6
-|: invoke-interface {v0, v6}, LIb/d;.resumeWith:(Ljava/lang/Object;)V
+|: invoke-virtual {v0, v6}, LBd/t;.resumeWith:(Ljava/lang/Object;)V
 |: monitor-exit v5

Attaching you the full diffs here: diff.zip

IzzySoft avatar Sep 01 '25 23:09 IzzySoft

> Configure project :
Checking task: :app:assembleFossRelease
Is full build: false

> Configure project :app
Full build detected, enabling Sentry Auth Token
Failed to load SENTRY_AUTH_TOKEN from local.properties: /build/repo/local.properties (No such file or directory)
Checking the license for package Android SDK Build-Tools 35 in /opt/sdk/licenses
License for package Android SDK Build-Tools 35 accepted.
Preparing "Install Android SDK Build-Tools 35 v.35.0.0".

This is warning, not a error, we can skip it

maxrave-dev avatar Sep 02 '25 04:09 maxrave-dev

@IzzySoft can i take a look on your build script

maxrave-dev avatar Sep 02 '25 04:09 maxrave-dev

Sure, here it is (logs are parallel to it in /logs/com.maxrave.simpmusic.json). Latest one is not up as the build was not yet committed, but the recipe did not change.

IzzySoft avatar Sep 02 '25 11:09 IzzySoft

For whatever reason, I've managed to get v0.2.21 RB – but only by building all ABIs incl. the universal APK,only to throw all but the arm64 away at the end. Somehow does not seem to work by building the arm64 only 🤷‍♂️

And seeing the linked issue: the curse of an integrated update checker, and one of the reasons why we suggest to leave them out. BUT: not sure how that's related to RB, as at IzzyOnDroid, your updates are simply published as-is, RB check runs on a separate track and does not keep updates from being shipped.

As I see no note in our metadata: is that update checker opt-in, and does it say where it checks for updates and what the implications of installing from there are (bypassing our additional screening)?

IzzySoft avatar Oct 05 '25 17:10 IzzySoft

If you need to build for a single, fixed ABI, some custom configuration is required in build.gradle.kts. You can see the relevant section here: https://github.com/maxrave-dev/SimpMusic/blob/102873073f8ccdcd4d1db93d9dcf9ff79dd1ab94/app/build.gradle.kts#L126-L132

If it's necessary for your workflow to build only the arm64 ABI, I can update the build.gradle.kts file to include a dedicated build variant for that, something like arm64foss, etc.

maxrave-dev avatar Oct 06 '25 02:10 maxrave-dev

Regarding the update checker, if you're not comfortable with it, I can add an IzzyOnDroid update channel in SimpMusic's settings. However, you'll need to provide me with instructions on how to fetch update information from IzzyOnDroid.

My thinking is that users who find the SimpMusic repository are generally not zero-knowledge and already understand the security implications of the application.

maxrave-dev avatar Oct 06 '25 02:10 maxrave-dev

If you need to build for a single, fixed ABI, some custom configuration is required in build.gradle.kts

Yeah, and if I do that, it's not RB anymore or, with the latest release, does not even build:

sed -r '/abi \{/,/}/ { s/include\(.*\)/include("arm64-v8a")/ ; s/isUniversalApk = true/isUniversalApk = false/ }' -i app/build.gradle.kts

Had that in before. Wasn't RB the last 2 versions. Didn't even build this time, complaining about conflicts with ndk filters versus ABI splits. I just removed that line, and it did the trick. A little overhead (building 4 additional APKs which are not needed), but we can pick the one we need.

If it's necessary for your workflow to build only the arm64 ABI, I can update the build.gradle.kts file to include a dedicated build variant for that, something like arm64foss, etc.

It's not strictly necessary; we basically try that adjustment to keep resource usage down. If you adjust that, but use your original workflow to build, we might run async again; so maybe we better leave it as-is for now – especially as it now seems to be working 😉 But thanks! And should you one day want this for yourself, a note is welcome when it's there of course.

Regarding the update checker, if you're not comfortable with it, I can add an IzzyOnDroid update channel in SimpMusic's settings. However, you'll need to provide me with instructions on how to fetch update information from IzzyOnDroid.

You can use our API for that, as described here in our wiki: Request https://apt.izzysoft.de/fdroid/api/v1/packages/com.maxrave.simpmusic and expect a JSON in returrn, listing all available versions – which for SimpMusic is exactly one:

{
  "packageName":"com.maxrave.simpmusic",
  "suggestedVersionCode":"40",
  "packages":[
    {"versionCode":"40","versionName":"0.2.21"}
  ]
}

My thinking is that users who find the SimpMusic repository are generally not zero-knowledge and already understand the security implications of the application.

That may be the case if they find it on a "Github browse spree" – but not when their F-Droid client shows it to them. Which is why the IzzyOnDroid App Inclusion Policy is very clear about this 😉

[the app] must not download additional executable binary files (e.g. addons, auto-updates, etc.) without explicit user consent. Consent means it needs to be opt-in (it must not be harder to decline than to accept or presented in a way users are likely to press accept without reading) and structured in a way that clearly explains to users that they’re choosing to bypass the checks performed in this repo if they activate it.

So how would you decide which update channel to check?

IzzySoft avatar Oct 06 '25 07:10 IzzySoft

So how would you decide which update channel to check?

Like Fdroid channel right now, are you ok with this?

maxrave-dev avatar Oct 06 '25 10:10 maxrave-dev

I don't know how "F-Droid channel" is figured, sorry 🙈 I can only tell if I understand… Also, are any changes on our end needed for that?

IzzySoft avatar Oct 06 '25 10:10 IzzySoft

Screenshot_20251006_182046_SimpMusic.jpg

Add one section IzzyOnDroid

maxrave-dev avatar Oct 06 '25 11:10 maxrave-dev

Ah! Now! Yes, that's fine. Is there a default set, or will it be queried at first start? And there's no "None" (e.g. if someone wants to keep that for their repo client)?

IzzySoft avatar Oct 06 '25 12:10 IzzySoft

It has "Disable auto check for updates" inside the app bro

maxrave-dev avatar Oct 07 '25 01:10 maxrave-dev

Then this is totally fine, thanks!

IzzySoft avatar Oct 07 '25 07:10 IzzySoft

Just wondering how you build the APK these days. According to your Github action, via build_and_sign_apk.sh – but I somehow doubt that, as that script calls ./gradlew composeApp:assemble"$BUILD_TYPE" and there are no build flavors anymore.

Now, I managed to build using

echo -e "\nisFullBuild=false" >> ./gradle.properties
./gradlew composeApp:assembleRelease

and it would even be RB, where it not for assets/sentry-debug-meta.properties containing a build timestamp:

#Generated by sentry-android-gradle-plugin
#Sun Nov 30 11:00:33 UTC 2025
io.sentry.build-tool=gradle

Can that be disabled for Sentry, at least when it's not a "FullBuild"?

IzzySoft avatar Nov 30 '25 20:11 IzzySoft

Sentry is not appear in apk which built. You can scan the dependencies. It is only gradle plugin, it mean it is only appear when build time, not runtime. I try some way to disable it from build time, but in now, I can not found anything

maxrave-dev avatar Dec 01 '25 05:12 maxrave-dev

I did not say Sentry itself was included – but that the mentioned asset is (you can check the APK yourself). And that is causing an issue. If that asset can be avoided entirely, even better.

IzzySoft avatar Dec 01 '25 08:12 IzzySoft

@IzzySoft Sorry guy, I see it now Image

I will check it and try to disable

maxrave-dev avatar Dec 01 '25 08:12 maxrave-dev