publish on F-Droid

[rhjdvsgsgks]

hi

Any plan to publish this application to F-Droid? Sagernet exists there before but was removed due to its discontinuation. Therefore, publishing a fork of Sagernet to F-Droid should not be too hard since upstream already meets the requirements. I guess the only thing missing is your consent to publish to F-Droid.

[dyhkwong]

If someone can help me realize reproducible build (so that the APKs published on F-Droid can be signed by myself) and other necessary procedures I will not oppose the inclusion. However, I have no interest in doing it on my own.

It seems that gomobile does not support reproducible build, I can switch to https://github.com/SagerNet/gomobile (like sing-box for Android) if necessary.

And how about those plugins (some are rarely maintained, some upstream developers has stopped development)? I don't think it is necessary to publish any plugins on F-Droid. Maybe we should only talk about the app itself?

[klzgrad]

should oppose redistribution

I don't oppose redistribution and the license permits it , but hopefully I have convinced you it is technically unnecessary and confusing for every plugin-integrating app to repackage it without substantial modification. I'm open to F-Droid packaging of these plugins for common use by SagerNet forks.

It was confusing in that SagerNet and NekoBox maintained the Naive plugin for a period of time and then due to circumstances they stopped maintenance of their apps and the plugin while the plugin core binary (libnaive.so) was being updated on schedule. After the second time I subsumed the plugin apk build code into main repo so that the maintenance of the plugin can be sustained.

[dyhkwong]

Let's not talking about reproducible build first. Are prebuilt binaries ever allowed on F-Droid? I think someone dismissed something when they published SagerNet on F-droid.

• geosite.dat and geoip.dat are binary files.
• To build geoip.dat a license key from MaxMind is required.
• MaxMind GeoIP (the source of geoip.dat) does not have a FOSS license.
• index.js (BrowserBridge) is obfuscated JavaScript, and its build procedure is much more complicated and outdated. I once tried to build BrowserBridge from source, it was not a happy experience. And BrowserBridge is a module of v2ray-core.
• Only index.js that matches the hash harcoded in v2ray-core is allowed to be loaded. (I use my own fork of v2ray-core so this restriction can be removed or modified.)

[linsui]

Data files are allowed and can even be non-free. But the geoip files should be from a legal source. IIUC the maxmind data doesn't allow redistribution. We can use other geoip data. This is what we do in NixOS.

Is that difficult to generate the BrowserBridge js file?

Since SFA is reproducible I thought we can make Exclave reproducible, too.

[dyhkwong]

Is that difficult to generate the BrowserBridge js file?

Is "browser bridge" considered to be a part of the code and need to be built from source? If so I'm afraid I need to choose not bundle it with the app and let users load it themselves. Building browser bridge from source requires very old toolchains and I don't know if it possible to produce the same hash as the official one.

[linsui]

Is "browser bridge" considered to be a part of the code and need to be built from source?

Is it used? Not sure if this is needed for mobile.

Is the very old toolchain go 1.19.3? It's not very old for debian. :) https://packages.debian.org/bookworm/golang-go If we can't rebuild the officiale file you can rebuild it and it should be easier to be reproducible.

[dyhkwong]

It is used on mobile ("use browser forwarding" of WebSocket) but it is an optional dependency. The official one was compiled with go 1.12.16 and never got updated. I can rebuild it with go 1.19, but need to modify the source code to remove the hardcoded hash restriction. It is user's responsibility not to install malware and sideload malware code.

[linsui]

Ah, now I understand what it is. I thought it should be built from source. I feel that it's better to rebuild it with go 1.19 and update the hardcoded hash.

[ghost]

https://github.com/zapstore/zapstore zap is decentralized store

Permissionless app store powered by your social network

[linsui]

@dyhkwong I'm trying to packag exclave as reproducible build. Let's solve the problems one by one.

There are embeded paths and ndk build-id in the libtermux.so. The build id can be removed with LOCAL_LDFLAGS := -Wl,--build-id=none, the paths can be removed with LOCAL_CFLAGS := -ffile-prefix-map=<NDK path>=ndk -ffile-prefix-map=<repo path>=Exclave. Could you please add that?

[dyhkwong]

There are embeded paths and ndk build-id in the libtermux.so.

Done in a91ec5d.

---

83a797e makes libgojni.so reproducible. However, there is still a strange diff in classes.dex. Maybe JDK version related?

├── smali/go/Seq.smali
│ @@ -2,21 +2,21 @@
│  .super Ljava/lang/Object;
│  .source "SourceFile"
│  
│  
│  # annotations
│  .annotation system Ldalvik/annotation/MemberClasses;
│      value = {
│ -        Lgo/Seq$RefMap;,
│ -        Lgo/Seq$GoRef;,
│ -        Lgo/Seq$GoRefQueue;,
│          Lgo/Seq$RefTracker;,
│ +        Lgo/Seq$GoObject;,
│ +        Lgo/Seq$GoRefQueue;,
│          Lgo/Seq$Ref;,
│ -        Lgo/Seq$Proxy;,
│ -        Lgo/Seq$GoObject;
│ +        Lgo/Seq$RefMap;,
│ +        Lgo/Seq$GoRef;,
│ +        Lgo/Seq$Proxy;
│      }
│  .end annotation
│  
│  
│  # static fields
│  .field private static final NULL_REFNUM:I = 0x29

This is the diffoscope result comparing the GitHub Actions build with my local build as of 83a797e. Assets are still in binary format.

[linsui]

Great, thanks! The build in F-Droid CI also has the same diff so it's reproducible with your local build. I saw the same diff on other apps which were caused by jdk version mismatch. But you already set it to 21 in https://github.com/dyhkwong/Exclave/blob/83a797e287b45b7b5e709de9a20ede9863162782/buildSrc/src/main/kotlin/Helpers.kt#L105 and I use the same jdk version. Is that possible gomobile somehow uses a different jdk? Could you please try setting the same jdk for libcore job?

[linsui]

Yep, it's reproducible now. :tada: Now let's solve the problem of geosite.dat and geoip.dat. The geosite.dat is totally fine. But could you please upload the version.txt files to release so that I can get the version from them directly? The geoip.dat is problemic though. In fact v2ray itself and even the debian package violate the license of MaxMind GeoIP data. We can replace the MaxMind data with https://db-ip.com/db/download/ip-to-country-lite then run the generator on it.

[dyhkwong]

check dev-assets branch.

But could you please upload the version.txt files to release so that I can get the version from them directly?

It seems that geosite.dat and geoip.dat version depends heavily on GitHub APIs. How about cebad914, or you have a better idea? It makes ./gradlew :app:downloadAssets download fixed version of assets according to geoip.version.txt and geosite.version.txt.

In fact v2ray itself and even the debian package violate the license of MaxMind GeoIP data.

Although I don't think compilation of objective facts like IP Country is protected by copyright, I created a DB-IP based geoip.dat repo and switched to it.

Toolchain (Go, JDK, NDK) version also needs to be pinned, and I am still figuring out how to compile browser forwarder automatically.

[linsui]

This is what I use to pin the assets version

sed -i -e '/geoip/s/latestRelease/getReleaseByTagName("202412120057")/' -e '/domain-list-community/s/latestRelease/getReleaseByTagName("20241210004721")/' buildSrc/src/main/kotlin/V2RayAssets.kt

I can get the go version from your CI config. Jdk version is hardly updated and it should only use LTS versions. You can also add ndk version in the CI config and I can extract it from there.

[linsui]

Looks like it's almost ready! Do you want to publish all 4 abis on F-Droid? F-Droid requires different apk has different version code, i.e., the version code of them should be something like 9331, 9332, 9333, 9334. Currently all of them have the same version code. And we need the metadata, as in https://github.com/SagerNet/SagerNet/tree/dev/metadata/en-US. Not sure if you want to also publish the plugins on F-Droid. It's better to use the Triple-T format so that we can add different metadata for the main app and the plugins. You can see an example in https://github.com/fcitx5-android/fcitx5-android/tree/master/app/src/main/play. See https://gitlab.com/-/snippets/1901490 for details.

[dyhkwong]

Do you want to publish all 4 abis on F-Droid

Yes. Metadata and different version code for different ABIs has been finished. I still want to add some extra code before publishing. Will ping you when I finish.

Not sure if you want to also publish the plugins on F-Droid.

I prefer no because of their large amount, and many of them are out of maintenance. If the author of naiveproxy want to publish naiveproxy plugin, this should happen on the upstream repo https://github.com/klzgrad/naiveproxy. I can provide some help if necessary.

[linsui]

Thanks!

[dyhkwong]

@linsui We can continue if you have time. I found some issues about extracting geosite.dat.xz and geoip.dat.xz and incorrectly overriding user downloaded ones, so now geosite.dat and geoip.dat (no .xz) are downloaded and bundled directly. Please take a look if there is still something missing.

[linsui]

IIUC it's all handled by the build scripts so it should work for F-Droid automatically? :)

[linsui]

Could you please make https://github.com/dyhkwong/LibSagerNetCore/blob/dec946cd1b7a2cc9a3ed7d613a3532aa29c3ad16/build.sh accept an arg to build a specific abi?

Ah, never mind. It seems it's still easier to patch the script directly.

[linsui]

I just tested the latest commit and it works well.

[dyhkwong]

So what's the next step? I can release some new versions to test.

I just tested the latest commit and it works well.

[linsui]

You just need to release a new version. :)

[dyhkwong]

Thanks! I just released a new version.

[linsui]

Ah, I forgot the DependencyInfoBlock problem.

It's a Signing block added by AGP and encrypted with the Google public key so it can't be read by anyone else except Google. You can read more about it here and here.

Could you please disable it with the following code?

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

[dyhkwong]

Ah, I forgot the DependencyInfoBlock problem.

Done in 0.13.1.

[linsui]

https://gitlab.com/fdroid/fdroiddata/-/merge_requests/17951 It will be built in the next cycle. It may take a week. Thanks for you help!

[linsui]

It has been published on https://f-droid.org/packages/com.github.dyhkwong.sagernet/ :tada:

You can add this in readme

[<img src="https://f-droid.org/badge/get-it-on.png"
    alt="Get it on F-Droid"
    height="80">](https://f-droid.org/packages/com.github.dyhkwong.sagernet/)

The Chinese version:

[<img src="https://f-droid.org/badge/get-it-on-zh-cn.png"
    alt="Get it on F-Droid"
    height="80">](https://f-droid.org/packages/com.github.dyhkwong.sagernet/)

[dyhkwong]

Thanks! Close as completed.