Bottles AppImage

[mirkobrombin]

cc @azubieta (GitHub doesn’t let me to assign the issue)

Bottles’ Appimage has been deprecated due to several issues. Let me make a small summary:

• packages like p7zip and cabextract are not always found even if they are present (in silverblue yes but Ubuntu and Arch no)
• WINE is unable to find winbind and 32bit dependencies (this may be related to the first point, i.e. that what is installed is not detectable by Bottles)
• Caffe (the default runner in Bottles) requires glibc 2.32 or higher
• for the first releases we used a custom script that did not include dependencies https://github.com/bottlesdevs/Bottles/blob/master/appimage-build.sh
• we switched to Appimage builder, this is the recipe which has the above problems https://github.com/bottlesdevs/Bottles/blob/master/AppImageBuilder.yml

Here is the list of necessary dependencies https://github.com/bottlesdevs/Bottles/wiki/Packaging#dependencies

[probonopd]

GitHub doesn’t let me to assign the issue

As far as I know, you can only assign to members of the repo/project, but @azubieta will see this.

[mirkobrombin]

Oh this makes sense 😅

[azubieta]

I've forked the project and done some fixes to the original recipe, now I was able to build it. Regarding to the issues:

• p7zip and cabextract are not always found, this may be because the execution flow goes out of the AppDir. To fix it you need to bundle env, bash, and any other interpreter that could be being used.

• Caffe (the default runner in Bottles) requires glibc 2.32 or higher. Same as above, but i guess that I would have to ~~make a modification to the AppRun to treat the caffe installation dir as part of the AppDir runtime~~ make caffe an AppDir/AppImage for itself.

• WINE is unable to find winbind and 32bit dependencies. Can you propose a testcase so I can evaluate what's wrong ?

My progress so far can be found at: https://github.com/azubieta/Bottles/tree/fix/appimage_package

[mirkobrombin]

Hi! I hope I understand everything well:

• Caffe: distributing runners as Appimage can be a problem because of how Bottles handle runners (it requires these as archives and does some edits). Do you think it is possible to add glibc as a recipe dependency as well?
• winbind: when it is not available, every single operation with WINE will result in 0130:err:winediag:ntlm_check_version ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution.

[azubieta]

AppImages created with builder already include a copy of glibc at APPDIR/runtime/compat at runtime we chdir into that folder and switch back before reaching the target application main so the binaries inside can be executed with the right libc. The issue is that we don't do that on purpose for binaries living outside the APPDIR (coffe in this case).

Regarding winbind, I'm not seen those messages so it "should" be fixed now.

[mirkobrombin]

Nice! I'll test your recipe

[azubieta]

Oh, the recipe is still broken because of the coffe issue.

[mirkobrombin]

Right. There is also another problem, libhandy. Bottles need 1.6 but Ubuntu 21.10 is stuck with 1.2. I think the Appimage can be built using any distro, right?

[azubieta]

You could use latest Debian if you want, also you can build libhandy from sources and manually deploy it.

[TheEvilSkeleton]

Wouldn't it be better to use Arch Linux as a base at this point?

[srevinsaju]

@mirkobrombin has the AppImage artifacts been removed from the older releases? I don't seem to find the older ones for some reason :(

[mirkobrombin]

@mirkobrombin has the AppImage artifacts been removed from the older releases? I don't seem to find the older ones for some reason :(

Hi! yes... we are working to make it works again. Old releases cannot be used due to Bottles repositories changes.

[mirkobrombin]

I'm trying to build the appimage with arch linux but an exception came up:

Traceback (most recent call last):
  File "/usr/bin/appimage-builder", line 33, in <module>
    sys.exit(load_entry_point('appimage-builder==1.0.0a3', 'console_scripts', 'appimage-builder')())
  File "/usr/lib/python3.10/site-packages/appimagebuilder/__main__.py", line 44, in __main__
    invoker.execute(commands)
  File "/usr/lib/python3.10/site-packages/appimagebuilder/invoker.py", line 29, in execute
    command()
  File "/usr/lib/python3.10/site-packages/appimagebuilder/commands/pacman_deploy.py", line 50, in __call__
    deployed_packages = pacman_deploy.deploy(
  File "/usr/lib/python3.10/site-packages/appimagebuilder/modules/deploy/pacman/deploy.py", line 87, in deploy
    os.symlink("usr/bin", appdir_root / "bin")
FileExistsError: [Errno 17] File exists: 'usr/bin' -> '/var/home/mirko/Projects/bottlesdevs/appimage/AppDir/bin'

those are the requested dependencies:

  pacman:
    include:
      - lib32-gnutls
      - gtksourceview4
      - hicolor-icon-theme
      - dconf
      - python
      - libhandy
      - gtk3
      - p7zip
      - python-gobject
      - python-requests
      - python-yaml
      - python-markdown
      - cabextract
      - webkit2gtk
      - xorg-xdpyinfo
      - gvfs
      - vkd3d
      - lib32-vkd3d
      - lib32-vulkan-icd-loader
      - vulkan-icd-loader
      - gamemode

[azubieta]

I thought I had fixed that bug, but it seems to strike again, hehehe. Please open an issue at appimage-builder.

The bottles recipe is almost ready. But we need to add an install step for Bottles modules. Basically we need to patch the interpreter paths to make the relative using this script: https://gist.github.com/azubieta/202c85f1f5a9e5c6be5cb066fcb2d5e2

[mirkobrombin]

Ok I'll send a report.

About modules, do you mean the python modules used by Bottles. Right?

[azubieta]

About modules, do you mean the python modules used by Bottles. Right?

I'm referring to all that goes inside $HOME/.local/share/bottles/

[mirkobrombin]

Exception reported in your repository 👀.

About modules, do you mean the python modules used by Bottles. Right?

I'm referring to all that goes inside $HOME/.local/share/bottles/

Those stuff are managed by Bottles itself on first run:

[azubieta]

The thing is that we need those binaries and scripts to be ran with the AppImage ld-linux and interpreters. To do so, the libapprun_hooks performs a chdir to the runtime directory inside the AppDir and executes the binaries from there if the paths are relative they will point to a file inside the AppDir and everything will work.

[mirkobrombin]

So will those files actually be inside the appimage?

[azubieta]

So will those files actually be inside the appimage?

No, they will live in the user home dir (or somewhere else). As I understood they are add-ons so the user can select which ones it will use. The inconvenient here is that once they are patched you will not be able to use then with other non-AppImage installation of Bottles.

[azubieta]

I just updated the recipe and you should be able to build a Bottles AppImage with the latest appimage-builder (the one in the git main branch)

[mirkobrombin]

So will those files actually be inside the appimage?

No, they will live in the user home dir (or somewhere else). As I understood they are add-ons so the user can select which ones it will use. The inconvenient here is that once they are patched you will not be able to use then with other non-AppImage installation of Bottles.

oh ok ok

I just updated the recipe and you should be able to build a Bottles AppImage with the latest appimage-builder (the one in the git main branch)

nice! I'll set up the github action too

[mirkobrombin]

I've updated the Github workflow according to your Dockerfile: https://github.com/mirkobrombin/Bottles/blob/master/.github/workflows/build_pkgs_unstable.yml

but it fails:

mkdir -p AppDir/usr/share/locale/$lang/LC_MESSAGES
msgfmt -o AppDir/usr/share/locale/$lang/LC_MESSAGES/bottles.mo po/$lang.po
done
cp -a data/icons AppDir/usr/share
mv AppDir/usr/share/icons/symbolic/scalable/apps/*.svg AppDir/usr/share/icons/hicolor/scalable/apps/
cp build/data/com.usebottles.bottles.desktop AppDir/usr/share/applications/
cp data/com.usebottles.bottles.gschema.xml AppDir/usr/share/glib-2.0/schemas/com.usebottles.bottles.gschema.xml
glib-compile-schemas AppDir/usr/share/glib-2.0/schemas/
cp -a data/com.usebottles.bottles.appdata.xml.in AppDir/usr/share/metainfo/com.usebottles.bottles.appdata.xml
mkdir -p $APPDIR/bin
ln -s bash $APPDIR/bin/sh
ln: failed to create symbolic link '/bin/sh': File exists
Traceback (most recent call last):
  File "/usr/local/bin/appimage-builder", line 8, in <module>
    sys.exit(__main__())
  File "/usr/local/lib/python3.10/dist-packages/appimagebuilder/__main__.py", line 44, in __main__
    invoker.execute(commands)
  File "/usr/local/lib/python3.10/dist-packages/appimagebuilder/invoker.py", line 29, in execute
    command()
  File "/usr/local/lib/python3.10/dist-packages/appimagebuilder/commands/run_script.py", line 69, in __call__
    raise RuntimeError("Script exited with code: %s" % _proc.returncode)
RuntimeError: Script exited with code: 1

[azubieta]

Use ln -sf to override any existent link also make sure that the target dir exists.

I created a docker image to ensure that the build environment is the same in my system that in Github as they tend to change stuff.

[mirkobrombin]

I’ve replicated your docker image in the workflow to avoid GitHub stuff and use the latest version of Ubuntu as in your 🤔

updated using -sf

[mirkobrombin]

🥳 https://github.com/bottlesdevs/Bottles/actions/runs/2162408389

[probonopd]

OK, I am a bit un an unusual setup (FreeBSD with Linuxulator and a Debian runtime) but I am getting:

FreeBSD% runappimage ./Bottles-devel-x86_64.AppImage 
AppDir                          com.usebottles.bottles.desktop  runtime
AppRun                          com.usebottles.bottles.svg      sbin
AppRun.env                      etc                             usr
bin                             lib                             var
ERROR: ld.so: object 'libapprun_hooks.so' from LD_PRELOAD cannot be preloaded (cannot stat shared object): ignored.
/tmp/appimage.CRe4qj03//usr/bin/python3.9: error while loading shared libraries: libm.so.6: cannot stat shared object: Invalid argument

Just fyi. more testing on my side needed.

[mirkobrombin]

What an interesting setup ahah Yes it definitely need more testing. On my side it raise some SSL errors but there may be other problems.

[azubieta]

ERROR: ld.so: object 'libapprun_hooks.so' from LD_PRELOAD cannot be preloaded (cannot stat shared object): ignored.

This error could be for two reasons the AppImage runtime wasn't removed on a external app call or we are missing a copy of libapprun_hooks.so for one arch.

/tmp/appimage.CRe4qj03//usr/bin/python3.9: error while loading shared libraries: libm.so.6: cannot stat shared object: Invalid argument

This could be a byproduct of the above error.

On my side it raise some SSL errors but there may be other problems.

In appimage-builder I bundle a copy of ssl certificates from the certifi package. This could be a solution here. Please share the system you are running to reproduce.

[probonopd]

...which might well be a byproduct of using the FreeBSD Linuxulator... how would I go about testing/debugging this systematically?