com.github.dan.nostoragerestrict icon indicating copy to clipboard operation
com.github.dan.nostoragerestrict copied to clipboard

Reproducible Builds

Open IzzySoft opened this issue 1 year ago • 18 comments
trafficstars

I've checked your app if its build is reproducible (see: Reproducible bulds, special client support and more in our repo), but I could not even build the app from the tag – as the tag points to a commit which is not present in the repo here. Did you maybe forget to push some code?

We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that.

Looking forward to your reply!

IzzySoft avatar Aug 02 '24 07:08 IzzySoft

I've checked your app if its build is reproducible (see: Reproducible bulds, special client support and more in our repo), but I could not even build the app from the tag – as the tag points to a commit which is not present in the repo here. Did you maybe forget to push some code?

We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that.

Looking forward to your reply!

I'm not very good at this code stuff, so I've never bothered with ensuring reproducible builds, I'll try when I have a new build to publish

DanGLVK avatar Aug 02 '24 15:08 DanGLVK

I'm not very good at this code stuff, so I've never bothered with ensuring reproducible builds

Fair enough – but hopefully the hints linked help you with that. Especially what's outlined there as §1 (aka "the cardinal rule"), which should be quite easy to follow.

I'll try when I have a new build to publish

Can't ask for more (except for a "ping" when that release is available, so I can check again). Thanks in advance for trying – and :crossed_fingers:

IzzySoft avatar Aug 02 '24 22:08 IzzySoft

No pressure, just wondering: any ETA for the new build?

IzzySoft avatar Nov 03 '24 10:11 IzzySoft

No pressure, just wondering: any ETA for the new build?

I mean, is there anything currently broken with the module to warrant an update? I haven't gotten any bug reports in months so i figured there's still no need for an update

DanGLVK avatar Nov 18 '24 05:11 DanGLVK

is there anything currently broken

The build process :wink: As I wrote above, I cannot even build from code. Reproducible builds are a security feature (if successful, they confirm the APK was really built from the code it claims to, nothing added or removed), which is really good to have especially for privileged apps.

I ask because I want to get my backlog down :see_no_evil: And I write "no pressure" as I know other people also have their load of tasks. So I carefully ask for an ETA to at least know where we stand – and in the hope it would not be too far into the future…

IzzySoft avatar Nov 18 '24 07:11 IzzySoft

is there anything currently broken

The build process 😉 As I wrote above, I cannot even build from code. Reproducible builds are a security feature (if successful, they confirm the APK was really built from the code it claims to, nothing added or removed), which is really good to have especially for privileged apps.

I ask because I want to get my backlog down 🙈 And I write "no pressure" as I know other people also have their load of tasks. So I carefully ask for an ETA to at least know where we stand – and in the hope it would not be too far into the future…

I'll see if I can get it done this week

DanGLVK avatar Nov 18 '24 11:11 DanGLVK

Thanks!

IzzySoft avatar Nov 18 '24 12:11 IzzySoft

Thanks!

Unfortunately, I can't guarantee I'll be able to push and update this week as my computer is unfortunately broken for the time being, and I can't afford to fix it at the moment, So I'm trying to ask my brother to see if he can help me fix it.

DanGLVK avatar Nov 22 '24 20:11 DanGLVK

Well, then it takes a week longer. Worse things happen these days…

IzzySoft avatar Nov 22 '24 22:11 IzzySoft

Well, then it takes a week longer. Worse things happen these days…

Got any way i can contact you more directly so i can have an easier time getting this whole reproducible build system working?

DanGLVK avatar Nov 27 '24 00:11 DanGLVK

Depends on what you have in mind. Email isn't really suited for this, nor are DMs on the Fediverse. Issues usually work best for this kind of task. Except when you're on a try-and-err run (as I am often when trying to get an app RB) and need to "debug online with a second pair of eyes" of course.

IzzySoft avatar Nov 27 '24 01:11 IzzySoft

Depends on what you have in mind. Email isn't really suited for this, nor are DMs on the Fediverse. Issues usually work best for this kind of task. Except when you're on a try-and-err run (as I am often when trying to get an app RB) and need to "debug online with a second pair of eyes" of course.

Alright, I'll talk to you tomorrow, just got my computer fixed tho it's kinda late rn, hope that's ok with you

DanGLVK avatar Nov 27 '24 01:11 DanGLVK

Hope your computer feels better meanwhile?

IzzySoft avatar Dec 05 '24 00:12 IzzySoft

Hope your computer feels better meanwhile?

Sorry, but I honestly don't know when I'll get around to it, I just have a hard time wrapping my head around on how to do this whole thing.

I don't mean to waste your time, so I'll just say this, I'll see if I can get around to doing it this week, If I don't do it by friday, you can just close this issue.

I'm honestly sorry for making you wait so long for seemingly no progress, I'm just not good at this whole development thing.

DanGLVK avatar Dec 05 '24 00:12 DanGLVK

Hey, don't worry – health first, get well! I'll just keep sending a ping every now and then if you don't mind. Will slow down and not ping more often than once a month (and of course fully stop once solved – or you tell me to).

Thanks for trying and all your help!

IzzySoft avatar Dec 05 '24 01:12 IzzySoft

Just wondering, will there be another release anytime soon™?

IzzySoft avatar Feb 08 '25 01:02 IzzySoft

Just wondering, will there be another release anytime soon™?

I'll see if I can come up with something, Frankly, it's hard to find a reason to update the module considering that, for the most part, it works just fine

DanGLVK avatar Feb 08 '25 01:02 DanGLVK

Understandable. But maybe we could at least see to get the code "in sync" here so I could try another round for building it and seeing if a) it can be built and b) it is reproducible? Doesn't need a new release right away; an APK built from a clean tree at a commit that is available here would suffice for that. You could attach that APK here to the issue (after renaming it to *.zip that is).

IzzySoft avatar Feb 08 '25 17:02 IzzySoft

So no chance, @DanGLES3?

hard to find a reason to update the module considering that, for the most part, it works just fine

Well… maybe that it says it's FOSS – but some of that FOSS code is missing as it was not pushed? 😉

IzzySoft avatar Apr 27 '25 13:04 IzzySoft

I mean, if it's not FOSS, we'd have to remove it from the IzzyOnDroid repo

IzzySoft avatar Jun 03 '25 14:06 IzzySoft

I mean, if it's not FOSS, we'd have to remove it from the IzzyOnDroid repo

What is missing? The repo is essentially a copy of my project folder

DanGLVK avatar Jun 03 '25 14:06 DanGLVK

If you look at the original/opening post of this issue: I tried to build from the same commit as your attached APK was built from – but that commit does not exist. It could of course mean you "squashed" after the build (so a new commit hash was generated) – but it could also mean the APK was built from a branch that is not public (e.g. was not pushed here).

No accusation meant here; but that's one thing RBs confirm, that the APK was indeed built from the source presented in the repo here. So a missing commit the app claims to have been built from, raises a red flag. Which is what I try to remove (the red flag / the doubt) since I've opened this issue almost a year ago.

Apologies for my insistence, but we want to make sure (to our best abilities) the apps we ship are really FOSS, as people trust us we'd do.

IzzySoft avatar Jun 03 '25 15:06 IzzySoft

If you look at the original/opening post of this issue: I tried to build from the same commit as your attached APK was built from – but that commit does not exist. It could of course mean you "squashed" after the build (so a new commit hash was generated) – but it could also mean the APK was built from a branch that is not public (e.g. was not pushed here).

No accusation meant here; but that's one thing RBs confirm, that the APK was indeed built from the source presented in the repo here. So a missing commit the app claims to have been built from, raises a red flag. Which is what I try to remove (the red flag / the doubt) since I've opened this issue almost a year ago.

Apologies for my insistence, but we want to make sure (to our best abilities) the apps we ship are really FOSS, as people trust us we'd do.

When I originally developed the no-storage-restrict module, it wasn't developed using a local git repo, so every time I released an update, I would just upload all the file changes directly to Github.

DanGLVK avatar Jun 03 '25 16:06 DanGLVK

Then what was the APK attached to tag 5.0 built from? If I follow the link to the commit it points to, this is what I get:

Image

The commit must have existed when you created the tag (probably locally). OK, let me see if I can "fool my way in". First try was a year ago, maybe I've learned a thing or two since then:

gradle verification fails, as the builder tries to check out the non-existing commit. So: turn off gradle verification, fetch the commit "timewise nearest to the tag", use our own gradle wrapper (which we didn't have "back then"), and see if that works…

git checkout 7df63f2bdf9ac9630e73fb0bffee77db5e413bde
sed -r '/signingConfig.getByName/d' -i app/build.gradle.kts  # we need an unsigned APK here
git clone https://codeberg.org/IzzyOnDroid/gradlew.py.git    # get our own wrapper
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 gradlew.py/gradlew.py --version 7.4.2 -v assembleRelease

Builds, but is not RB. There are differences in classes.dex.Dex diff says I have some stuff in there your APK does not have. But it has versionCode='5' versionName='0.5.0', which was only set with that commit and didn't exist before. The next commit was done 5 months after the APK was built, and only removes the main.zip file. So that APK was built from "somewhere shortly before that commit", either from a "dirty tree with local changes" – or from that not (no longer?) existing commit the tag points to. I have no chance to reproduce that APK – and thus to confirm it was really built from this source code here.Which is why I thought a new release with a clean build from an existing commit would be the best way to solve this.

diffs.zip

IzzySoft avatar Jun 03 '25 21:06 IzzySoft

Then what was the APK attached to tag 5.0 built from? If I follow the link to the commit it points to, this is what I get:

Image

The commit must have existed when you created the tag (probably locally). OK, let me see if I can "fool my way in". First try was a year ago, maybe I've learned a thing or two since then:

gradle verification fails, as the builder tries to check out the non-existing commit. So: turn off gradle verification, fetch the commit "timewise nearest to the tag", use our own gradle wrapper (which we didn't have "back then"), and see if that works…

git checkout 7df63f2bdf9ac9630e73fb0bffee77db5e413bde sed -r '/signingConfig.getByName/d' -i app/build.gradle.kts # we need an unsigned APK here git clone https://codeberg.org/IzzyOnDroid/gradlew.py.git # get our own wrapper JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 gradlew.py/gradlew.py --version 7.4.2 -v assembleRelease Builds, but is not RB. There are differences in classes.dex.Dex diff says I have some stuff in there your APK does not have. But it has versionCode='5' versionName='0.5.0', which was only set with that commit and didn't exist before. The next commit was done 5 months after the APK was built, and only removes the main.zip file. So that APK was built from "somewhere shortly before that commit", either from a "dirty tree with local changes" – or from that not (no longer?) existing commit the tag points to. I have no chance to reproduce that APK – and thus to confirm it was really built from this source code here.Which is why I thought a new release with a clean build from an existing commit would be the best way to solve this.

diffs.zip

I'll see what I can do this week.

DanGLVK avatar Jun 04 '25 13:06 DanGLVK

Friendly heads-up – not sure which week "this" is 🙈

IzzySoft avatar Jul 12 '25 00:07 IzzySoft

Friendly heads-up – not sure which week "this" is 🙈

Frankly, I currently have no interest in pursuing reproducible builds, so you may remove my module from your repo if you so choose

I deeply apologize for wasting your time

DanGLVK avatar Jul 12 '25 03:07 DanGLVK

Thanks for the clear words, Dan! I totally understand. And no worries, we all "miscalculate" when it comes to our "wishlists" (ask me! And things I thought to do in one free afternoon end up taking weeks…).

I took it off my backlog now. Feel free to close this issue then – and give me a ping should you pick up the idea at some point in the future (if closed, this issue could be reopened in that case then).

IzzySoft avatar Jul 12 '25 12:07 IzzySoft