OpenOrienteering
Build and upload AppImage for each git push
This PR, when merged, will compile this application on Travis CI upon each git push, and upload an AppImage to a temporary download URL on transfer.sh (available for 14 days). The download URL is toward the end of each Travis CI build log of each build (see below for how to set up automatic uploading to your GitHub Releases page).
For this to work, you need to enable Travis CI for your repository as described here prior to merging this, if you haven't already done so.
Providing an AppImage would have, among others, these advantages:
- Applications packaged as an AppImage can run on many distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
- One app = one file = super simple for users: just download one AppImage file, make it executable, and run
- No unpacking or installation necessary
- No root needed
- No system libraries changed
- Works out of the box, no installation of runtimes needed
- Optional desktop integration with
appimaged - Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
- Can optionally GPG2-sign your AppImages (inside the file)
- Works on Live ISOs
- Can use the same AppImages when dual-booting multiple distributions
Here is an overview of projects that are already distributing upstream-provided, official AppImages.
Please note: Instead of storing AppImage builds temporarily for 14 days each on transfer.sh, you could use GitHub Releases to store the binaries permanently. This way, they would be visible on the Releases page of your project. This is what I recommend. See https://docs.travis-ci.com/user/deployment/releases/. If you want to do this for continuous builds, also see https://github.com/probonopd/uploadtool.
If you would like to see only one entry for the Pull Request in your project's history, then please enable this GitHub functionality on your repo. It allows you to squash (combine) the commits when merging.
If you have questions, AppImage developers are on #AppImage on irc.freenode.net.
Thank you very much for your contribution.
Since we already use OBS for building Linux and Windows packages, it would make sense to use this service instead of Travis-CI. You got this covered, so I made my attempt an attempt to build there. It took longer than expected, but a proof of concept is done: https://build.opensuse.org/package/show/home:dg0yt:branches:home:dg0yt:staging/openorienteering-mapper-unstable
The results apply to any Mapper AppImage, no matter if from OBS or from Travis-CI:
- The resulting AppImage is large (54 MB now at OBS), much more than our other image-style packages. This is due to more numerous and complex dependencies, especially for Qt Assistant/WebKit and for gdal. This can only be reduced if we build the libraries ourselves again. Well, we have got the tool (superbuild).
- The licensing of all the bundled libraries needs to be reviewed, and documentation needs to be added to the AppImage as needed. We have got the Infrastructure ("licensing provider" stuff in doc/licensing), but is a lot of work, given >100 libraries.
- Where Mapper uses resources which are distributed with the application, we need to make sure that it picks up resources from the AppImage. Also resources for Proj.4 and GDAL need to be handled.
While we could ignore the size, that last two issues must be fixed before distribution.
BTW, I added set -e to the top of my script. I wonder if this shouldn't be the default: break early instead of ignoring errors.
Great work @dg0yt. I tested your AppImage and it launched well for me on openSUSE LEAP 42.0. Since it took you longer than expected, which were the things you think could be improved/streamlined?
Regarding PCRE,
- openSUSE (as of 2017) lacks libpcre.so.3 but provides libpcre.so.1
- Ubuntu (as of 2017) lacks libpcre.so.1 but provides libpcre.so.3
Why they cannot agree on one is unbeknownst to me. In any case, I will change linuxdeployqt such as to no longer exclude this library, for this reason. Thanks.
Why it took a little bit longer:
- I couldn't make my local
obs build AppImage x86_64 ...work (on Ubuntu). - I tried both building from source code and from existing rpm ('ingredients').
- First time building from source, I probably run into a temporary OBS issue.
- Then attempting to build from existing rpm, I got way to many files in the package, e.g. all gdal binaries, and many directories which the app image doesn't need.
- Then building from source again, I had to get used to the script-inside-YAML syntax, especially how to deal with breaking long lines.
- Then I tried to get compiler flags right, as I do in other OBS builds. It is usually easy in the system's native build system. But AppImage is alien. (That's the
export CFLAGS=$(rpm --eval %{optflags} | sed -e "s/ -g / /")) - At some point in time, I managed to confuse the OBS source service which parses the .yml file.
- Finally I stumbled over missing libpcre.so.1. I had to learn that linuxdeployqt uses a hard-wired blacklist which cannot be changed by command line arguments or configuration files. But just copying the file before running the tool works... (Solution came from an issue on Github.)
In brief, especially for OBS, the toolchain still has some rough edges, and there is lack of best practice examples.
We started using Azure Pipelines for CI where we can use a hosted Ubuntu 16.04. I wouldn't mind porting the script to Azure pipelines, but I don't know about the portability of Appimages generated on this version of Ubuntu, and I would appreciate if there were existing Azure build scripts to use as a guideline. We may also be able to use Docker containers but I didn't try something like this yet.
I can merge into a stand-by branch if you want to remove your fork of the repository. But I see no point in merging this to an active branch as long it is not used (for building) and maintained.
- We don't use Travis CI but Azure Pipelines.
- Licenses of included files have not been reviewed. There were also libraries which I would prefer not to distribute.
- Unless also using our Superbuild, versions of included libraries are inconsistent with both our other packages and the Linux system where the software is run. I don't want to support such a configuration.
Thanks for your response, of course I'd like to see this not only merged but also actually used, so that AppImages finally become available after all that time.
We don't use Travis CI but Azure Pipelines.
Can we use Travis CI for the AppImages and Azure Pipelines for the rest?
Licenses of included files have not been reviewed.
Who needs to review them and when will this happen?
There were also libraries which I would prefer not to distribute.
Which ones, and why?
Unless also using our Superbuild, versions of included libraries are inconsistent with both our other packages and the Linux system where the software is run. I don't want to support such a configuration.
Where can I learn about the Superbuild and how can it be used?
Where can I learn about the Superbuild and how can it be used?
@probonopd, take a look here:
- https://github.com/OpenOrienteering/superbuild
So, if I understand it correctly someone would essentially need to port
https://github.com/OpenOrienteering/mapper/pull/898/files#diff-354f30a63fb0907d4ad57269548329e3R20-R29
to
https://github.com/OpenOrienteering/superbuild/blob/master/ci/azure-pipelines.yml#L60-L74
Correct?