Native openSUSE package does not run appimaged

[probonopd]

The native openSUSE package by @marguerite currently installs /usr/bin/appimaged-x86_64.AppImage and appimagetool-x86_64.AppImage.

To get appimaged to work, one has to

appimaged-x86_64.AppImage --install
# This gives an error, so we do the next step by hand:
cp /usr/bin/appimaged-x86_64.AppImage /home/linux/.local/bin/appimaged

By handling installation and/or packaging this a bit differently, this could be made automatic. I will write more about this when I have time to think about this. I need to provide additional information.

[probonopd]

Looking at https://build.opensuse.org/package/show/home:MargueriteSu/appimagekit it seems like you are not merely re-packaging the prebuilt AppImages, but are really compiling everything from source. Congratulations, great work.

If you are doing this, then you might as well want to install the tools natively into the system, i.e., not as an AppImage, and use the openSUSE mechanism for launching a per-user daemon (using systemd?) to start appimaged. That way the user would not have to deal with appimaged-x86_64.AppImage --install.

Packaging appimaged inside an AppImage is more meant for users who cannot use native rpm or deb packages.

Please let me know if the above is not clear, then I will try to explain in more detail.

Again, thank you for your great work, well done @marguerite.

[marguerite]

Hi, @probonopd

At first I did want to package appimagetool and appimaged directly into the system, but as I know little about the inside information, I don't know which files are actually needed. eg AppRun? And I don't know what appimaged-X86_64.AppImage --install will do.

Actually I just finished packaging, didn't research on the details in real life packaging yet. But as you can see, I created two AppImages in my build, which indicates my build works.

In my build, I statically compiled openssl, xz and inotify-tools because most of the times such static compiled libraries will not be provided by a distribution, and patched some CFLAGS/LDFLAGS to use them. If you agree, I can tweak the patch to upstream...

[probonopd]

Hi @marguerite. Sorry my response took a bit longer due to Christmas holidays.

There are different ways to distribute software (among others):

1. As an AppImage. This is intended for users who do not want or cannot use a package manger, and does not install things to /usr (non-root users cannot write there at all). An example for this way of packaging appimaged is appimagetool-x86_64.AppImage. AppImages are normally relocatable, which means the user can out an AppImage at every location in the filesystem (e.g., even on an USB drive) and it will still work. While this works great for regular applications, it is a challenge for a daemon that is expected to be launched automatically each time the system is started. For this reason, I added appimaged --install which copies the AppImage to a fixed location, namely $HOME/.local/bin/appimaged and installs an autostart desktop file in $HOME/.config/autostart/appimagekit-appimaged.desktop(source code).

AppRun is needed in this case as the "entry point" into the AppImage, so that the AppImage "knows" which binary inside the AppImage should be executed when the AppImage is launched.

2. As native distribution packages. For example, a deb or RPM. This usually means installing software to a $PREFIX like /usr, and to use dependencies on other packages (e.g., libraries) provided by the distribution. Since native distribution packages are installed by a package manager that has root rights, they can install stuff into /usr and don't need the tricks AppImage uses to be relocatable. An example for this way of packaging appimaged is appimaged_1.0_amd64.deb.

/usr/bin/appimaged
/usr/lib/systemd/user/appimaged.service
/usr/share/doc/appimaged/...

Note that no AppRun is needed in this case. Also note that libraries like libarchive should not be part of such a package; instead, a dependency on the distribution's libarchive package should be set.

The challenge here is how to get systemd to launch appimaged as a per-user daemon when the user logs into a session. I think it needs a /usr/lib/systemd/user/appimaged.service file like this:

[Unit]
Description=AppImage daemon
After=basic.target

[Service]
ExecStart=/usr/bin/appimaged
Restart=always
RestartSec=5s
StartLimitInterval=0

[Install]
WantedBy=graphical.target

But how do we enable this for all users in the RPM package's %post section? I could not quite make it work yet, but maybe you will find out how to do this. I must admit that I am not an RPM packaging expert at all.

# Enable per-user launchd unit for all users in a way that users can disable it
systemctl --global enable appimaged # I can't seem to get it to work on Ubuntu

# Enable per-user launchd unit for all users # I can't seem to get it to work on Ubuntu
mkdir -p /usr/lib/systemd/user/graphical.target.wants/
( cd /usr/lib/systemd/user/graphical.target.wants/ ; ln -s ../appimaged.service . )

In any case, the result should be that one appimaged process is running when a user logs in, as can be seen using pidof appimaged. This process should be running under the user's account (not as root). There are many other processes (e.g., in GNOME) that want to be run this way, so it should definitely be doable.

3. As a mixture of both. For example, an AppImage being packaged inside a deb or RPM. I do not recommend it.

So @marguerite I would recommend to use 2. for your packages. Please let me know if this explanation is not clear, I will try to do my best to make it as understandable as possible.

[marguerite]

Thanks for the explanation. I'll play with my specfile tonight. will see what we can get.

[hideout]

and this is just the beginning at archlinux with appimages ... its also one of my favorite linux distris

[jgmdev]

@probonopd I just fixed the PKGBUILD for archlinux using latest stuff on the git repository: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=appimage-git

You said the following:

# Enable per-user launchd unit for all users in a way that users can disable it
systemctl --global enable appimaged # I can't seem to get it to work on Ubuntu

# Enable per-user launchd unit for all users # I can't seem to get it to work on Ubuntu
mkdir -p /usr/lib/systemd/user/graphical.target.wants/
( cd /usr/lib/systemd/user/graphical.target.wants/ ; ln -s ../appimaged.service . )

I wrote a workaround to enable the appimaged service for each user on the post install script as found here that may give you an alternative idea on how to accomplish it: https://aur.archlinux.org/cgit/aur.git/tree/appimage-git.install?h=appimage-git

I found a segmentation fault issue on the appimaged binary, it seems that it is having issues with recursing on all the directories that it uses to search for appimage files:

appimaged -v
Unrecognized file '/opt/firebird/bin/fbguard'
get_thumbnail_path: /home/myuser/.cache/thumbnails/normal/d2a8d51ab6c8b0a9716975d2c7e562bd.png
get_thumbnail_path: /home/myuser/.cache/thumbnails/normal/9f854c9e292d1a7fd3664ae96911865a.png
opendir error
[1]    14923 segmentation fault (core dumped)  appimaged -v

when running it with valgrind the most important part of the errors is this:

==17268== Thread 1:
==17268== Invalid read of size 4
==17268==    at 0x661F7E9: readdir (in /usr/lib/libc-2.24.so)

Maybe I should open a new issue for this, let me know.

[jgmdev]

so I decided to take a look at the appimaged.c source file and discovered the issue is because opendir is failing on a directory with no read permission and this isn't handled.

[marguerite]

@probonopd

Hi, I am still confused about whether we should package AppRun or not.

For the appimaged/appimagetool itself, it is not needed since AppRun is the entry point for AppImage format, we don't need it if we don't distribute appimaged/appimagetool via AppImage format.

But, appinagetool is an application that creates AppImage from AppImage Dir. AppImage Dir is not a normal directory, it will be squashfs that needs "boot codes", that is, AppRun.

So I think if we want appimagekit to be "out of the box" for the end users, we will need a helper that creates a skeleton AppImage Dir. If we don't distribute AppRun, will appimagetool insert such thing into a normal directory automatically? Apparently not, as I see, there's no such "initialize" feature in appimagetool right now. If we don't distribute AppRun, I think there's no way for the end users to get it from other places because it is built here. If end users have to build appimagetool anyway to get something, our packaging actually fails to achieve the goal.

So I am thinking that I should write a bash script to make it happen... and you said AppRun is not needed...so I am confused...

[probonopd]

Hi @marguerite

Generating that AppDir (=the ingredients that go into an AppImage) is currently outside the scope of appimagetool. Some projects use a shell script called AppRun, others compile AppRun.c, yet others use a precompiled AppRun binary from GitHub Releases. If you would like to package it, I am not against it, but it should not be in a directory that is on the $PATH, because excecuting it e.g., from the system's /usr/bin will not do something useful.

With "AppRun is not needed" I was referring to the fact that in order to run appimaged and appimagetool, when packaged natively rather than as an AppImage, no AppRun is involved (because it is only needed in AppImages not native packages). Of course you need some sort of AppRun if you want to put together an AppDir for converting it into an AppImage.

[probonopd]

Thanks @jgmdev

Have you seen something like https://aur.archlinux.org/cgit/aur.git/tree/appimage-git.install?h=appimage-git to be used for other packages as well? Do you think this is the way we are supposed to do enable per-user systemd services in package postinstall/prerm scripts? How are recent GNOME versions packaged, which apparently also use per-user systemd services?

[marguerite]

@probonopd

Thanks, now I clearly know what I should do to modify my specfile

[marguerite]

@probonopd

This issue can be closed now. see https://build.opensuse.org/package/show/devel:tools:building/appimagekit

[jgmdev]

@probonopd Haven't seen this method used elsewhere (which now I improved to use systemctl directly for enabling/activation). What I can say is that this way the appimaged service is activated for every user after installation and the user has the option to disable it if he wants to.

I have been inspecting the /usr/lib/systemd/user/ directory and I see some services are marked as static and they are of type dbus, which I guess dbus takes care of launching them, but im not sure how this works, would need to research if the dbus scenario applies to appimaged case.

[marguerite]

openSUSE defined some amazing RPM macros for systemd：https://build.opensuse.org/package/view_file/Base:System/systemd-rpm-macros/macros.systemd?expand=1

hope it can help. see systemd_user_post systemd_user_preun

[jgmdev]

Thanks @marguerite! What I don't understand is why those macros are using --global in combination with --user. Maybe there is something I'm missing from the systemctl documentation.

[probonopd]

Great progress, thank you, will test very soon @marguerite.

[probonopd]

Using http://download.opensuse.org/repositories/devel:/tools:/building/openSUSE_Factory/x86_64/appimaged-7~git20161206.0720f5f-3.1.x86_64.rpm on openSUSE-Tumbleweed-GNOME-Live-x86_64-Snapshot20160921-Media.iso, I got errors in YaST regarding an invalid signature but installed nonetheless. Then I logged out of my session and logged in again. I expected that appimaged would be running by now, but it was not.

linux@nohostname:~> systemctl --user status appimaged
● appimaged.service - AppImage daemon
   Loaded: loaded (/usr/lib/systemd/user/appimaged.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

I can start it manually:

linux@nohostname:~> systemctl --user start appimaged
linux@nohostname:~> systemctl --user status appimaged
● appimaged.service - AppImage daemon
   Loaded: loaded (/usr/lib/systemd/user/appimaged.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2017-01-24 18:30:43 UTC; 1s ago
 Main PID: 10174 (appimaged)
   CGroup: /user.slice/user-999.slice/[email protected]/appimaged.service
           └─10174 /usr/bin/appimaged

Now, when I download https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage using the browser, it is automatically registered with the system and can be launched via the GNOME Shell.

So, appimaged is working but only after manually invoking a systemctl command. Is there a way around this?

[probonopd]

Now, http://download.opensuse.org/repositories/devel:/tools:/building/openSUSE_Factory/x86_64/appimagetool-7~git20161206.0720f5f-3.1.x86_64.rpm on openSUSE-Tumbleweed-GNOME-Live-x86_64-Snapshot20160921-Media.iso:

# Extract an AppImage
linux@nohostname:~> /home/linux/Downloads/Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage --appimage-extract
linux@nohostname:~> rm squashfs-root/.DirIcon

# Package an AppImage using appimagetool
linux@nohostname:~> appimagetool squashfs-root/ -n

# Verify that it has worked
linux@nohostname:~> ls -lh *AppImage
-rwxr-xr-x 1 linux users 226K Jan 24 18:36 Leafpad-x86_64.AppImage
linux@nohostname:~> ./Leafpad-x86_64.AppImage 

Works! So, appimagetool is working beautifully.

The only thing I wonder, the tiny runtime executable that gets injected into each generated AppImage, could we compile this on a very old system and use that? If we don't, then the AppImages generated e.g., on a Tumbleweed system might not run on (much) older systems due to dependencies on recent symbols. (For this reason we compile AppImageKit on CentOS 6.)

[marguerite]

@probonopd

I think openSUSE intends not to start the service automatically...you see I used the official macro...sometimes users install things just for later use, eg I installed mysql but I didn't want it to take my resources until I finished writing my blog service. It should be their choice to enable some daemon or it sounds like spyware. for system default enabled services, there is a central place/package if I remember correctly.

there is surely something we can do, we can just 'systemctl --user enable' it in post and stop & disable it in preun. or we just make it included in that central package.

about AppRun, I understand your concern. Then it is not good by distributing apprun.c or AppRun because they are all compiled on the developers' systems eg openSUSE TW or Archlinux. The only right thing is to compile it on an outdated system eg rhel4/sles10...

[probonopd]

@marguerite as I understand it, Gnome these days also runs (and enables by default) per-user systemd services, maybe you can find out how it is done there, and use the same for appimaged?

Good point about AppRun, too.

[marguerite]

@probonopd can you please tell me which upstream package includes such service? then I can find openSUSE one. Yesterday I tried but didn't find such examples.

[probonopd]

@marguerite I think http://software.opensuse.org/package/gvfs-backends contains some, since it has service files that get installed to /usr/lib/systemd/user/.

However I can't seem to find anything in the postinstall script that activates them:

rpm -qp --scripts  gvfs-backends-1.30.3-1.1.x86_64.rpm 
postinstall scriptlet (using /bin/sh):

/usr/bin/glib-compile-schemas /usr/share/glib-2.0/schemas
postuninstall scriptlet (using /bin/sh):

if [ $1 -eq 0 ]; then                                           
  /usr/bin/glib-compile-schemas /usr/share/glib-2.0/schemas  
fi

[marguerite]

@probonopd

openSUSE installed the user service but did nothing. I think it is not enabled at all. I also checked Debian, it didn't install the user service at all. So from the distribution point of view either they want users to do it, or they didn't find a way of doing it yet :-(

[probonopd]

Could it be that it gets activated on demand using dbus or something like that?

[probonopd]

@marguerite do you see any way the postinstall script could enable the service? So that the user does not have to do it by hand? Then we could close this issue. Thanks!

What do you think about the solution proposed by @jgmdev https://aur.archlinux.org/cgit/aur.git/tree/appimage-git.install?h=appimage-git

[probonopd]

@jgmdev

so I decided to take a look at the appimaged.c source file and discovered the issue is because opendir is failing on a directory with no read permission and this isn't handled.

Thanks for finding this out. Please do open a separate issue and/or PR. Thanks!

[marguerite]

@probonopd the Arch way is doable, but not perfect. It will enable the service for all local users that have /home/xxx directories, even if he doesn't login at all. I think start/stop the service for LOGUSER is enough, if a user logout will stop all user services (which I don't know). Anyway starting service for an inactive user sounds like redundancy or security risk.

And why daemon-reload when appimaged is still running? I think daemon-reload should be the last step for stop. and starting the service doesn't require a reload at all

[probonopd]

It will enable the service for all local users that have /home/xxx directories, even if he doesn't login at all.

If I understand it correctly, "enable" only means that it will launch once the user logs in. If the user does not log in, it doesn't get started. Perhaps I am wrong though.

[probonopd]

Just discovered that this spec file by @adrianschroeter also produces a package for appimaged... https://build.opensuse.org/package/view_file/OBS:AppImage/AppImageKit/AppImageKit.spec

[probonopd]

Maye related: https://github.com/systemd/systemd/issues/2690