noten-lernen icon indicating copy to clipboard operation
noten-lernen copied to clipboard

[Feature Request] publish to F-Droid

Open ulu opened this issue 10 months ago • 23 comments

I like to see your famous app to be released on https://f-droid.org. IMHO its the main app store after Google's PlayStore.

ulu avatar Apr 04 '24 12:04 ulu

What do you think about this @MelvilQ?

offa avatar Jun 02 '24 14:06 offa

There was a request more than one year ago: https://gitlab.com/fdroid/rfp/-/issues/2076

I don't know why it isn't accepted yet and what I could do to make it happen. Also I do not have a GitLab account to comment there. @IzzySoft Can you help?

MelvilQ avatar Jun 02 '24 20:06 MelvilQ

Sorry, I'm currently not active at F-Droid. As for IzzyOnDroid: The app is already listed there.

IzzySoft avatar Jun 03 '24 07:06 IzzySoft

I don't know why it isn't accepted yet and what I could do to make it happen. Also I do not have a GitLab account to comment there.

Someone needs to do the actual work and submit the build metadata. If you do not have any objects against F-Droid submission, I can give it a try.

@IzzySoft seriously sad news :cry:

offa avatar Jun 03 '24 15:06 offa

seriously sad news 😢

Yeah, and was never properly communicated, or the underlying issue addressed. I had hopes I could have returned once that happens (solving the issue etc). But … well, I won't go into details here. Let's stick with the summary: it never happened. And I see no chance it ever will, not in the foreseeable future.

But then, Melvil's app is welcome to stay at IzzyOnDroid – which will also stay around. And blossom, I hope – we have "big plans" there :wink:

IzzySoft avatar Jun 03 '24 19:06 IzzySoft

Yeah, and was never properly communicated, or the underlying issue addressed. I had hopes I could have returned once that happens (solving the issue etc). But … well, I won't go into details here. Let's stick with the summary: it never happened. And I see no chance it ever will, not in the foreseeable future.

Sorry for hijacking this issue, but this are really bad news and quite a loss for the F-Droid community. :speak_no_evil:.


Topic related: I have tried to add the F-Droid build metadata, but the build fails at npm i (called within cordova/):

+ npm i
added 89 packages, and audited 90 packages in 2s
9 packages are looking for funding
  run `npm fund` for details
1 moderate severity vulnerability
To address all issues, run:
  npm audit fix
Run `npm audit` for details.
2024-06-03 20:03:49,503 DEBUG: Updating main project
2024-06-03 20:03:49,504 DEBUG: Directory: build/de.melvil.noten/cordova
2024-06-03 20:03:49,504 DEBUG: > /opt/android-sdk/tools/android update lib-project -p .
Error: . is not a valid project (AndroidManifest.xml not found).
2024-06-03 20:03:49,806 ERROR: Could not build app de.melvil.noten: Failed to update project at .
==== detail begin ====
Error: . is not a valid project (AndroidManifest.xml not found).
==== detail end ====
2024-06-03 20:03:49,806 DEBUG: Error encountered, stopping by user request.

offa avatar Jun 03 '24 20:06 offa

I am not able to reproduce this error with a fresh git clone on Windows.

MelvilQ avatar Jun 04 '24 17:06 MelvilQ

As far as I can tell, this F-Droid related and npm i works fine. I am still searching for the underlying issue.

offa avatar Jun 04 '24 17:06 offa

The project builds now :+1:. Still some steps to go :-).

It's not reproducible yet, since there's a signature mismatch: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/15172 (full build log accessible there).

I'm no Cordova dev, but could the different cordova-plugin-vibration versions in config.xml and package.json cause the issue?

offa avatar Jun 06 '24 18:06 offa

I'm no Cordova dev, but could the different cordova-plugin-vibration versions in config.xml and package.json cause the issue?

I'm no such dev either, but the answer is: Yes, of course! Not just "could", but definitely "would". They include different files, which prevents reproducibility.

IzzySoft avatar Jun 06 '24 23:06 IzzySoft

@MelvilQ the build gets stuck after these steps:

[...]
                             static/samples/piano/9.mp3     383 kB          [emitted]  [big]  
                            static/samples/piano/45.mp3     159 kB          [emitted]         
                                                  sw.js    1.17 kB          [emitted]         
  Build complete.
  Tip: built files are meant to be served over an HTTP server.
  Opening index.html over file:// won't work.

Do you have an idea?

offa avatar Jun 09 '24 17:06 offa

What do you mean by "build gets stuck"? I looked at the log and have a few vague assumptions.

The last 2 lines tell me it was canceled by a person not by a build error:

                         static/samples/piano/7.mp3     351 kB          [emitted]  [big]  
                         static/samples/piano/8.mp3     351 kB          [emitted]  [big]  
                         static/samples/piano/9.mp3     383 kB          [emitted]  [big]  
                        static/samples/piano/45.mp3     159 kB          [emitted]         
                                              sw.js    1.17 kB          [emitted]         
  Build complete.
  Tip: built files are meant to be served over an HTTP server.
  Opening index.html over file:// won't work.
Terminated
==== detail end ====
2024-06-07 08:27:08,938 DEBUG: Error encountered, stopping by user request.
WARNING: script canceled externally (UI, API)

But before this we have a first build run:

Terminated
2024-06-07 08:27:08,938 ERROR: Could not build app de.melvil.noten: Error running prebuild command for de.melvil.noten:1.8.1

Seems there is something missing in our build environment. Which would explain warnigns like this:

(node:5289) Warning: Accessing non-existent property 'cat' of module exports inside circular dependency

This seems to tell there is something missing and we have some circular dependencies which is always a bad sign. The latter maybe comes from multiple warnings like the following. Maybe we have the wrong packages installed or missing some in our build environment?

W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:1 and /etc/apt/sources.list.d/debian.sources:1

And in the end I see the following. Any software build shall stop if critical vulnerabilities are detected.

156 vulnerabilities (6 low, 56 moderate, 49 high, 45 critical)

ulu avatar Jul 05 '24 19:07 ulu

The last 2 lines tell me it was canceled by a person not by a build error:

Yes, the build didn't progress some time so it was eventually killed by hand. This is reproducible within the CI build.

The build is executed on a Debian based system, with additional software installed within the build:

  • OpenJDK 11
  • npm
  • Cordova (via npm)
  • build-tools 31.0.0
  • build-tools 30.0.3

The build recipe is available here: https://gitlab.com/fdroid/fdroiddata/-/blob/779abc986184334a80d44ae42c5daeb61ea2400b/metadata/de.melvil.noten.yml

Do you know what packages are required on the build host to build the project? Or maybe some versions above aren't ok?

offa avatar Jul 07 '24 13:07 offa

You are using two different versions of build tools. Why? Most recent in Debian is 29.0.3+9. Most recent version on android.com is 34.0.0.

Latest OpenJDK version is 22.

You are using a Debian 11 which is oldstable.

I will try to build on a Debian 12 with latest Android-SDK and latest OpenJDK. Then we will see if this makes a difference. But this will takes some time...

ulu avatar Jul 09 '24 19:07 ulu

You are using two different versions of build tools. Why? Most recent in Debian is 29.0.3+9. Most recent version on android.com is 34.0.0.

Due some kind of try-and-error :sweat_smile:.

Latest OpenJDK version is 22.

Newer versions didn't work and the build had a build requirement to Java 11 for Cordova. I'm no Cordova dev so error on my side may still be a reason.

You are using a Debian 11 which is oldstable.

OS version is defined by F-Droid build environment. It's possible to install packages, so I wouldn't expect this as an issue.

offa avatar Jul 10 '24 17:07 offa

Both oldstable and stable Debian VMs are not happy when insalling cordova:

npm install -g cordova
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '[email protected]',
npm WARN EBADENGINE   required: { node: '>=16.13.0' },
npm WARN EBADENGINE   current: { node: 'v12.22.12', npm: '7.5.2' }
npm WARN EBADENGINE }
...

Even if I follow the install options on this repo I failed to build build the webapp and the app.

I will lookout how to solve this mess within the next days.

ulu avatar Jul 11 '24 20:07 ulu

Success to build the project in a Manjaro VM. I had to install latest version of nodejs and npm:

pacman -S nvm
nvm install --lts

After that I was able to build and run the web application on http://localhost:8080.

So I installed missing requirements for Android apk:

pacman -S cordova
pacman -S jdk-openjdk
pacman -S android-tools

But I still failed to build the android apk because some paths are missing like:

[email protected] copy-locales
cp -r ./locales/android/values-en ./platforms/android/app/src/main/res/

cp: das Verzeichnis './platforms/android/app/src/main/res/' kann nicht angelegt werden: Datei oder Verzeichnis nicht gefunden

If I generate the path I get another error:

Could not load API for android project /<path>/noten-lernen/cordova/platforms/android/cordova/Api.js

I will try to reproduce this in Debian oldstable. From this we have to deal with android-sdk.

ulu avatar Jul 12 '24 19:07 ulu

Now I was able to build the web app on Debian oldstable by using the same steps as in Manjaro. Seems to be an issue of cordova, openjdk and Android SDK. I will investigate this further.

ulu avatar Jul 12 '24 19:07 ulu

I had a lot of work to get a recent OpenJDK and Android SDK to run. What a mess in Debian...

You need to install a more recent version of JDK (LTS) to be able to use sdkmanager (openjdk-17-jdk of oldstable Debian does not fullfill our dependencies).:

wget https://download.java.net/java/GA/jdk22.0.1/c7ec1332f7bb44aeba2eb341ae18aca4/8/GPL/openjdk-22.0.1_linux-x64_bin.tar.gz

tar xvf openjdk-22.0.1_linux-x64_bin.tar.gz
sudo mv jdk-22.0.1 /usr/lib/jvm/openjdk-22

check installed java and you will not see your recent java version

sudo update-alternatives --list java

Let Debian know about our recent JDK version:

sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/openjdk-22/bin/java 20
sudo update-alternatives --config java

Setting the Java environment, create a new source script.

sudo tee -a  /etc/profile.d/jdk22.sh<<EOF
export JAVA_HOME=/usr/lib/jvm/openjdk-22
export PATH=\$PATH:\$JAVA_HOME/bin
EOF

sudo chmod 755 /etc/profile.d/jdk22.sh

Source to effect the change in your current shell session.

source /etc/profile.d/jdk22.sh

Check Java version to verify it works.

java --version
openjdk 21.0.2 2024-01-16
OpenJDK Runtime Environment (build 21.0.2+13-58)
OpenJDK 64-Bit Server VM (build 21.0.2+13-58, mixed mode, sharing)

echo $JAVA_HOME
/usr/lib/jvm/openjdk-22

Then I installed codova:

npm install -g cordova

After that I had fun with android tools in Debian.

apt install android-sdk*

On my system the paths are not set. So I exported the Android SDK paths

sudo tee -a  /etc/profile.d/androidSDK.sh<<EOF
export ANDROID_HOME=/usr/lib/android-sdk
export PATH=\$PATH:\$ANDROID_HOME/cmdline-tools/bin
export PATH=\$PATH:\$ANDROID_HOME/platform-tools
EOF

Then do

source /etc/profile.d/androidSDK.sh

After this I had got an eror from sdkmanager:

sdkmanager --list
Error: Could not determine SDK root.
Error: Either specify it explicitly with --sdk_root= or move this package into its expected location: <sdk>/cmdline-tools/latest/

I read in the internet that this can be circumvented by

mv /usr/lib/android-sdk/cmdline-tools/* /usr/lib/android-sdk/cmdline-tools/tools/.

WTF?

So I had to change one line in my /etc/profile.d/androidSDK.sh:

export PATH=\$PATH:\$ANDROID_HOME/cmdline-tools/tools/bin

Again do a

source /etc/profile.d/androidSDK.sh

After that we can run sdkmanager

sdkmanager --list
...
Available Updates:
  ID             | Installed | Available
  -------        | -------   | -------  
  platform-tools | 28.0.2    | 35.0.1 

After that I installed the version I found in your build script:

sdkmanager --update
sdkmanager "platforms;android-31" "build-tools;31" "extras;google;m2repository" "extras;android;m2repository"

But now I still fail with cordova because either it cannot create a path:

:~/noten-lernen/cordova# npm run build

> [email protected] build
> npm run copy-locales && cordova build android --release -- --packageType apk


> [email protected] copy-locales
> cp -r ./locales/android/values-en ./platforms/android/app/src/main/res/

cp: das Verzeichnis './platforms/android/app/src/main/res/' kann nicht angelegt werden: Datei oder Verzeichnis nicht gefunden

And when I create this path I stuck at:

:~/noten-lernen/cordova# mkdir -p ./platforms/android/app/src/main/res/
:~/noten-lernen/cordova# npm run build

> [email protected] build
> npm run copy-locales && cordova build android --release -- --packageType apk


> [email protected] copy-locales
> cp -r ./locales/android/values-en ./platforms/android/app/src/main/res/

Could not load API for android project ./noten-lernen/cordova/platforms/android/cordova/Api.js

I lack experience with cordova. Do you have an idea how to solve this issues to be able to build the apk?

ulu avatar Jul 13 '24 21:07 ulu

F-Droid uses Debian bookworm currently.

linsui avatar Jul 14 '24 09:07 linsui

Thank you for this information. The Debian version is not my problem. Cordova is unable to build even with all fullfilled requirements. It seems cordova is unable to create necessary files and paths inside this project.

ulu avatar Jul 14 '24 19:07 ulu

@MelvilQ @offa Do you have an idea in which configuration files I can find the entries for the required software component versions (npm, cordova, node) for a build environment? I think we cannot go any further without this information because npm ci succeeds only for web build but cordova always fails for apk.

ulu avatar Jul 26 '24 18:07 ulu

I think some of the build components depend on the Cordova version (listed here), which seems configured in the package.json.

offa avatar Jul 30 '24 16:07 offa