Paperize icon indicating copy to clipboard operation
Paperize copied to clipboard

[Bug] F-Droid reproducible build failed

Open linsui opened this issue 1 year ago • 14 comments

https://gitlab.com/fdroid/fdroiddata/-/jobs/8058432296

Somehow aboutlibraries doesn't work in your build. Could you please take a look? Thanks!

linsui avatar Oct 11 '24 05:10 linsui

I believe it's fixed in v2.3.1, let me know.

Anthonyy232 avatar Oct 11 '24 19:10 Anthonyy232

2024-10-12 04:31:15,913 ERROR: Could not build app com.anthonyla.paperize: Unexpected version/version code in output; APK: 'paperize-vproperty(java.lang.String, map(java.lang.String provider(?) check-type()))' / '33', Expected: '2.3.1' / '33'

The version name is broken...

linsui avatar Oct 12 '24 06:10 linsui

Is this incorrect? image

Maybe there's was a weird cache error, I'll replace the APK in the release I guess

Anthonyy232 avatar Oct 12 '24 07:10 Anthonyy232

Your apk is correct. But my build has the broken version name. Any idea?

linsui avatar Oct 12 '24 07:10 linsui

Not really sure. Some things I changed this release were: JDK to 17 NDK to 28.0.12433566 BuildTools to 35.0.0

Lastly, maybe this?

applicationVariants.all {
        this.outputs
            .map { it as com.android.build.gradle.internal.api.ApkVariantOutputImpl }
            .forEach { output ->
                val apkName = "paperize-v${this.versionName}.apk"
                output.outputFileName = apkName
            }
    }

It's not immediately obvious to me that this would be the issue, but I can try rolling that part back. What do you think?

Edit** Actually seems like I forgot to push something. Are you able to test it based on the latest code in the main branch, or do I need to make a new release?

Edit** I'm pretty sure I might've fixed the issue and can make a new release perhaps?

Anthonyy232 avatar Oct 12 '24 07:10 Anthonyy232

Yes, I thought it caused by this. Will test it soon. :)

linsui avatar Oct 12 '24 07:10 linsui

The version name issue is fixed but the apk is not reproducible.

https://gitlab.com/linsui/fdroiddata/-/jobs/8068812928 Your apk is built from the old commit. And could you please downgrade the ndk to a stable version? We can't use a beta version.

linsui avatar Oct 12 '24 08:10 linsui

Sure. I'll do that tomorrow

Edit** Done

Anthonyy232 avatar Oct 12 '24 08:10 Anthonyy232

Looks like you didn't use NDK when building the apk. The problem is that the prof file is not reproducible. I don't know why this happens. :shrug:

linsui avatar Oct 13 '24 08:10 linsui

Weird, I haven't really done anything different with regards to the NDK other than up'ed it. I downgraded it to the last stable version (27.1.12297006) before building the APK. I guess I'll try downgrading to the last known working version and then make a temporary release for you to test if possible?

Anthonyy232 avatar Oct 13 '24 08:10 Anthonyy232

It shouldn't be related to ndk. Since you didn't use NDK I didn't use it and the native libs are reproducible. But the prof file is not reproducible somehow.

linsui avatar Oct 13 '24 08:10 linsui

I don't think any of these would likely be it, but maybe I'm mistaken. I'll list a few things I changed recently and let me know if any of them may (or known) cause problems. Happy to revert them to test: JVM 1.8 -> 17 buildToolsVersion 34 -> 35 Old plugin system -> version catalog

applicationVariants.all {
        this.outputs
            .map { it as com.android.build.gradle.internal.api.ApkVariantOutputImpl }
            .forEach { output ->
                val apkName = "paperize-v${this.versionName}.apk"
                output.outputFileName = apkName
            }
    }

Adding baseline profile module and baseline-prof.txt, startup-prof.txt

Could also just do a completely clean APK build on my side if it helps, I'm not sure if it's my side (computer) that's causing the issue or the project itself for the mismatch in APK. I don't really know a ton about the building process.

Anthonyy232 avatar Oct 13 '24 08:10 Anthonyy232

Adding baseline profile module and baseline-prof.txt, startup-prof.txt

Since the baseline.prof file is not reproducible this sounds the most related.

Maybe we can test with a clean build first. :)

linsui avatar Oct 13 '24 08:10 linsui

Okay, clean build done as a pre-release, let me know if that works. Otherwise I will revert the baseline stuff

Anthonyy232 avatar Oct 13 '24 19:10 Anthonyy232

https://gitlab.com/linsui/fdroiddata/-/jobs/8074904278 The prof file is still different.

linsui avatar Oct 14 '24 05:10 linsui

Okay, reverted the baseline profile stuff and clean build with files pulled from GitHub. Hope that works, otherwise I'm not too sure.

Anthonyy232 avatar Oct 14 '24 08:10 Anthonyy232

https://gitlab.com/linsui/fdroiddata/-/jobs/8076095976 Weird... I'll try rebuilding old versions.

linsui avatar Oct 14 '24 08:10 linsui

2.1.0 is still reproducible. Since 2.3.0 the prof is not reproducible anymore.

linsui avatar Oct 14 '24 11:10 linsui

The prof file dumped from your apk:

entry=assets/dexopt/baseline.prof
prof version=010 P
num_dex_files=1
dex_data_header 0
  profile_key='classes.dex'
  num_type_ids=2510
  dex_checksum=0xb0ac8ad4
  num_method_ids=30287
dex_data 0
  num_hot_method_ids=8578
  num_type_ids=2510
  bitmap_size=7572
entry=assets/dexopt/baseline.profm
profm version=002
num_dex_files=1
profile_idx=0
  profile_key='classes.dex'
  num_type_ids=6776
  num_class_ids=2510

The one from my build:

entry=assets/dexopt/baseline.prof
prof version=010 P
num_dex_files=1
dex_data_header 0
  profile_key='classes.dex'
  num_type_ids=2510
  dex_checksum=0xb0ac8ad4
  num_method_ids=30287
dex_data 0
  num_hot_method_ids=8579
  num_type_ids=2510
  bitmap_size=7572
entry=assets/dexopt/baseline.profm
profm version=002
num_dex_files=1
profile_idx=0
  profile_key='classes.dex'
  num_type_ids=6776
  num_class_ids=2510

so the only different is the num_hot_method_ids.

linsui avatar Oct 14 '24 11:10 linsui

We can disable the prof file.

tasks.whenTaskAdded {
    if (name.contains("ArtProfile")) {
        enabled = false
    }
}

linsui avatar Oct 14 '24 11:10 linsui

2.1.1 is still reproducible. 2.2.0 has the prof file problem.

linsui avatar Oct 14 '24 12:10 linsui

--verbose diff for baseline.prof of 2.1.0

$ diff upstream.log ci.log
3,4c3,4
< uncompressed_data_size=46931
< compressed_data_size=6708
---
> uncompressed_data_size=46935
> compressed_data_size=6713
8c8
<   hot_method_region_size=34312
---
>   hot_method_region_size=34316
12c12
<   num_hot_method_ids=8578
---
>   num_hot_method_ids=8579
5362a5363,5364
>   num_inline_caches=0
>   method_id=5488

licaon-kter avatar Oct 14 '24 12:10 licaon-kter

https://github.com/Anthonyy232/Paperize/compare/v2.1.1...v2.2.0

maybe the update to gradle 8.10.1? eg. https://docs.gradle.org/8.10.1/release-notes.html#configuration-cache-improvements clone a clean tree and clean all caches before building?

updated agp to 8.6.1? https://developer.android.com/build/releases/past-releases/agp-8-6-0-release-notes#android-gradle-plugin-8.6.1

licaon-kter avatar Oct 14 '24 12:10 licaon-kter

Current AGP is actually 8.7, I'll downgrade it back to 8.6.0 and we can test that first, if that doesn't work then I'll downgrade Gradle as well and put out a pre-release as a test

Anthonyy232 avatar Oct 14 '24 17:10 Anthonyy232

Pre-release version is out, let me know

Anthonyy232 avatar Oct 17 '24 21:10 Anthonyy232

https://gitlab.com/linsui/fdroiddata/-/jobs/8119823783 It's not enough...

linsui avatar Oct 18 '24 04:10 linsui

Ok, downgraded Gradle, hope that works in the new prerelease

Anthonyy232 avatar Oct 18 '24 05:10 Anthonyy232

It works now. :)

linsui avatar Oct 18 '24 07:10 linsui

Well, cool, good to know haha

Anthonyy232 avatar Oct 18 '24 08:10 Anthonyy232

I tested locally:

  • 8.10 is ok
  • 8.10.1 is NOT OK, run once ok; run 10 more builds: 2 ok, 8 NOT ok
  • 8.10.2 is NOT OK https://docs.gradle.org/8.10.2/release-notes.html

/LE: amend after more testing

licaon-kter avatar Oct 18 '24 11:10 licaon-kter