audiveris icon indicating copy to clipboard operation
audiveris copied to clipboard

Also make linux releases!

Open Altonss opened this issue 2 years ago • 82 comments

Hello, I just discovered the project and am very interested! The project seems amazing and I want to help bringing the software on linux. I saw we can already build it from source and that's already great. But It would be even better provide builds, for example through AppImages or Flatpaks :) If I find the time to help with this issue, I would be happy to contribute!

Altonss avatar Dec 06 '21 11:12 Altonss

Sure, our small team would be happy to benefit from such contribution!

Personally I develop on Windows and my competency about Linux is close to zero. So I will make sure that Max, by far our most competent Linux guy, is aware of your proposal.

I know nothing about AppImages and Flatpaks. I think I remember that Oracle is to provide a general binary-packaging utility in some next Java release (perhaps based on jlink?). A track to explore. We just have a Windows-specific installer today, a more general approach would be interesting of course.

hbitteur avatar Dec 06 '21 11:12 hbitteur

I made some progress to package Audiveris as a flatpak. For the moment it is using the build obtained using gradle and wraps this into a flatpak. That's not very clean, but at least it's a beginning. Should I share my recipe? I could also try something with the jar files, but that's more complicated as I would need gradle to run these...Or am I wrong?

Altonss avatar Dec 06 '21 17:12 Altonss

Does Audiveris needs network access? Because I got some errors because of some lacking network access

Altonss avatar Dec 06 '21 17:12 Altonss

Audiveris checks periodically for a potential new release. This is the only automatic network access I can think of (and you can change the polling period which is weekly by default). If there is no network, you should get just a warning like "Could not connect to Audiveris project".

Of course if you manually select, in the Help menu, items like Handbook, Wiki, Web Site, Check for Update, you will try to reach the information on the network.

hbitteur avatar Dec 06 '21 17:12 hbitteur

Regarding flatpak, I can't help you. I will let Max answer your question if he can.

hbitteur avatar Dec 06 '21 17:12 hbitteur

I've created a flatpak recipe here. I created a helper tool that allows downloading all Java dependencies and using them for a clean flatpak build using grade --offline. I have created 4 branches for packaging 5.2.4, 5.2.5, and the current master and development branches, respectively. These branches contain all required dependencies, so they can be used for a simple flatpak-builder invocation. The bootstrapping procedure for downloading the dependencies is explained in the README. There are also two slim branches with no dependencies (scripts and scripts-17 for openJDK11 and openJDK17, respectively).

Network access works in my package. But opening a browser to read the docs currently doesn't work, this seems to be a generic Java/flatpak issue.

@Altonss, I just discovered this issue here after I was already done with my work. I hope you're not upset. Perhaps we can make a joint effort to get this ready for prime-time?

@hbitteur, would you agree to submitting these packages to flathub?

mwilck avatar Apr 05 '22 19:04 mwilck

Thanks a lot for your work @mwilck ! I already had tried to build the flatpak, and it kind of worked, but thanks for having made a clean version. Are all the functionnalities working? It would be sooo cool to be able to release some day on Flathub :)

Altonss avatar Apr 05 '22 19:04 Altonss

This is still in it's early stages. I haven't checked if everything works (as I said, launching a browser does not). I've opened some sample music sheets and run OMR on it; I was quite satisfied with the results so far. I'd be very grateful if you could give my packages a try. You should be able to build them fairly easily (clone my repo, checkout branch b5.2.5, and run flatpak-builder --repo /some/dir builddir org.audiveris.audivers.yml).

mwilck avatar Apr 05 '22 20:04 mwilck

I've created a short Wiki page explaining what the various branches are.

mwilck avatar Apr 05 '22 20:04 mwilck

The web browser problem is solved now due to help I got in the openJDK17 issue.

mwilck avatar Apr 09 '22 20:04 mwilck

@hbitteur: I'd appreciate a comment if you'd agree with submitting this to flathub, and if not, what you think I (we) should do beforehand. In particular, would you object me submitting this with the flatpak application ID org.audiveris.audiveris?

mwilck avatar Apr 09 '22 20:04 mwilck

@mwilck I have no objection to your submitting. And I'll be out for a couple of days so don't wait for other comments from my side, simply keep going as you intend.

hbitteur avatar Apr 09 '22 22:04 hbitteur

I haven't submitted to flathub yet, but I've reworked the build scripts such that no dependencies need to be stored locally. Everything is now downloaded by flatpak-builder before gradle builds offline.

mwilck avatar Apr 15 '22 23:04 mwilck

Any progress on this topic? We do need such Linux releases!

hbitteur avatar Jun 29 '23 09:06 hbitteur

Sorry I haven't been able to follow up here. I suppose my previous packaging work could be brought back to work with relatively little effort. I will try to find some time to work on it soon.

FYI, back in 2022 when I worked on this, I got frustrated because Audiveris would always crash in unpredictable ways in the text recognition stage, typically with references to freed memory, which AFAICT should never happen in Java. I am certain that this wasn't caused by the flatpak, it also happened with native binaries, so maybe it was an issue with Tesseract, or possibly with my JRE (actually various different JREs I tried). I didn't want to submit something I knew was going to crash on non-trivial scan attempts.

I guess I should have opened in issue here, but I thought that this can't be a generic issue but must be caused in some ways by my Linux environment. I didn't make any progress with the bug analysis though, and eventually gave up, especially as the project for which I had intended to use Audiveris was finished.

mwilck avatar Aug 20 '23 19:08 mwilck

So ... I had another look at my build environment for the flatpak, and to my surprise, I found that it still works smoothly on the lastest code. I have a branch for packaging Audiveris 5.3.1, and the development branch has all the meta data for packaging current Audiveris development. The other good news is that I couldn't reproduce the Tesseract crashes I'd previously seen.

So it seems that we're actually in quite decent shape. I'm sure there are still lots of issues (for example, the "About" page shows wrong versions, not idea why).

mwilck avatar Sep 29 '23 22:09 mwilck

@hbitteur:

The flatpak organization prefers that the author of an application submits the package to Flathub. Maybe you could consider doing the submission yourself? The procedure is described here.

You need a fork and a clone of flathub itself, which must be populated with the necessary files to build the Audiveris package. I have created a sample flathub fork here ("audiveris" branch). It was created from my Audiveris-flatpak repo, branch b5.3.1, using the script flathub-copy.sh.[^gen]

AFAICS you could use my flathub fork for a first submission attempt of Audiveris 5.3.1 without changes. However, please review it again, in particular the org.audiveris.audiveris.metainfo.xml file. If you find issues, or have any additional questions, contact me. Please also have a look at the Flathub submission requirements page, licensing etc.

Note that Flathub's requirements wrt licensing aren't overly strict. In my understanding, if we can package a Windows binary under some open-source license and distribute it, we should be able to publish on Flathub, too. But IANAL.

If you can't do this submission yourself for whatever reason, let me know, and I'll try to figure out how I can do it in your name.

[^gen]: The purpose of the mwilck/Audiveris-flatpak code is to create a build manifest for building Audiveris offline, as required by flatpak. The code that will eventually be submitted to Flathub is generated by the code in the mwilck/Audiveris-flatpak repo.

mwilck avatar Sep 29 '23 22:09 mwilck

@mwilck

It's good news. Both for the Audiveris project and for all those who ask for easy installation on Linux.

We can add the fact that, thanks to recent PR from Peter Greth, Audiveris was able to replace JPodRenderer with Apache PDFBox. JPodRenderer still had a dependency on JAI (Java Advanced Imaging) which was not fully open-source. The road is now clear.

Today I went through the Flatpak and Flathub documentations that you pointed out in your post. I also read the 15 files contained in your Flathub fork. Should I say that I was far from understanding every line?

So, I think you, Martin, are the person who should take the lead on this Flatpak/Flathub submission. Indeed, I read that "some applications are distributed on Flathub without the involvement of their developers" and that Flathub "would prefer that these applications be controlled by their authors". But that's not our case. I think all Audiveris contributors, starting with me, would appreciate this submission.

This Audiveris project is open in all directions, all contributions are welcome. Baruch contributed with the Windows installer. This Linux distribution will be your contribution.

I will try to help you as much as I can. For example, I think we should aim for a new version (5.3.2), due to some bug fixes I'd like to include.

We'll get into more detail in the coming days, but I wanted to tell you that you should go for it. Thank you very much for your work. /Hervé

P.S. I need to resurrect my VirtualBox installation to get my hands on the Linux environment again

hbitteur avatar Sep 30 '23 19:09 hbitteur

Flathub submission created. Let's see how it goes. @hbitteur please have a look and let me know if anything is not as you would like it.

mwilck avatar Oct 23 '23 21:10 mwilck

@hbitteur, have you ever tried running Audiveris on other platforms than x86_64? The flatpak build fails for ARM because of missing tesseract libraries, and I wonder if it's worth the effort to try and make it build. For now I am just trying to disable the ARM build.

mwilck avatar Oct 25 '23 19:10 mwilck

@mwilck No, I have never tried it myself. For what it's worth, have a look at issue #564

hbitteur avatar Oct 26 '23 13:10 hbitteur

We made it! Audiveris is now on flathub. :rocket:

I suppose this issue can be closed.

mwilck avatar Oct 30 '23 10:10 mwilck

@hbitteur, perhaps you want to add a note about this in your installation guide.

mwilck avatar Oct 30 '23 10:10 mwilck

Sure, I need to do this! Let me try to run the installation on my PC.

Thanks a lot!

hbitteur avatar Oct 30 '23 10:10 hbitteur

Thanks a lot @mwilck ! Once it is in the installation guide, maybe it would be a good thing to verify it on flathub https://docs.flathub.org/docs/for-app-authors/verification

Altonss avatar Oct 30 '23 10:10 Altonss

I haven't double-checked, but I suppose verification can be only done for a single account. I can do this, but I'd leave the decision to @hbitteur whether he wants to link the app to his own account instead, which would increase credibility.

mwilck avatar Oct 30 '23 10:10 mwilck

@Altonss, perhaps you could give the package a test?

mwilck avatar Oct 30 '23 11:10 mwilck

About the verification: the process requires me create a file with an UUID-like token in a file on the audiveris.org web page (https://audiveris.org/.well-known/org.flathub.VerifiedApps.txt). I obviously can't do that myself. So @hbitteur, I'll need your cooperation for creating that file. Perhaps it's actually easier if you do this yourself. You can use github to identify to flathub.

(edited) Well it seems that audiveris.org doesn't currently do https at all. So I guess this needs more patience.

mwilck avatar Oct 30 '23 11:10 mwilck

@Altonss, perhaps you could give the package a test?

Tried it, and so far everything worked fine! Only issue is, by default Export book exports to /home/username/.var/app/org.audiveris.audiveris/data/AudiverisLtd/audiveris/piece/piece.mvt1.mxl, which isn't ideal for the end user I think (it would be better in documents for ex).

Altonss avatar Oct 30 '23 13:10 Altonss

@hbitteur, perhaps you want to add a note about this in your installation guide.

Wrote first little documentation for the flatpak installation https://github.com/Audiveris/audiveris/pull/687.

Altonss avatar Oct 30 '23 13:10 Altonss