Ensure local linuxdeploy plugins are executable

[rmartin16]

Issue

If a user specifies a local file without execute permissions as a linuxdeploy plugin, linuxdeploy will ignore the plugin.

linuxdeploy Log Excerpt

DEBUG: Searching for plugins in directory /app/appimage/new
DEBUG: File/symlink is not executable, skipping: /app/appimage/new/linuxdeploy-plugin-gtk.sh
DEBUG: Entry is a directory, skipping: /app/appimage/new/new.AppDir
DEBUG: File/symlink is not executable, skipping: /app/appimage/new/briefcase.toml
DEBUG: File/symlink is not executable, skipping: /app/appimage/new/Dockerfile 

Note: these entries only show in linuxdeploy's log with the -v0 flag and DEBUG_PLUGIN_DETECTION env var non-null.

Additionally, plugins at that are AppImages (whether user-provided or downloaded) causes errors in Docker either because their ELF header is not patched or they are not extracted before they are run.

Resolution

After the plugin is copied in to the app's bundle directory, set the execute permissions for the file. Plugins specified via URL already have execute permissions added after the download.

All AppImage files now have their ELF header patched. The --appimage-extract-and-run flag for linuxdeploy is replaced with APPIMAGE_EXTRACT_AND_RUN=1 environment variable so all AppImages are extracted before use.

PR Checklist:

• [x] All new features have been tested
• [x] All new features have been documented
• [x] I have read the CONTRIBUTING.md file
• [x] I will abide by the code of conduct

[rmartin16]

Ahhh....I didn't realize plugins could be an AppImage themselves. Nonetheless, when I try to use a local version of the Qt plugin or just specify qt as a plugin in pyproject.toml right now, linuxdeploy errors out.

>>> Running Command:
>>>     docker run --volume /home/user/tmp/beeware/newapp/linux:/app:z --volume /home/user/.cache/briefcase:/home/brutus/.cache/briefcase:z --rm --env PATH=/home/brutus/.cache/briefcase/tools/linuxdeploy_plugins/qt:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --env VERSION=0.0.1 --env DISABLE_COPYRIGHT_FILES_DEPLOYMENT=1 --env DEBUG_PLUGIN_DETECTION=1 briefcase/com.example.newapp:py3.10 /home/brutus/.cache/briefcase/tools/linuxdeploy-x86_64.AppImage --appimage-extract-and-run --appdir=/app/appimage/new/new.AppDir -d /app/appimage/new/new.AppDir/com.example.newapp.desktop -o appimage -v0 --deploy-deps-only /app/appimage/new/new.AppDir/usr/app_packages/cairo --deploy-deps-only /app/appimage/new/new.AppDir/usr/app_packages/gi --deploy-deps-only /app/appimage/new/new.AppDir/usr/lib --deploy-deps-only /app/appimage/new/new.AppDir/usr/lib/python3.10/lib-dynload --plugin qt
>>> Working Directory:
>>>     /home/user/tmp/beeware/newapp/linux
linuxdeploy version 1-alpha (git commit ID 287c436), GitHub actions build 110 built on 2022-08-17 20:56:20 UTC
DEBUG: Searching for plugins in directory /home/brutus/.cache/briefcase/tools 
DEBUG: Entry is a directory, skipping: /home/brutus/.cache/briefcase/tools/linuxdeploy_plugins 
DEBUG: Doesn't match plugin regex, skipping: /home/brutus/.cache/briefcase/tools/linuxdeploy-x86_64.AppImage 
DEBUG: Searching for plugins in directory /tmp/appimage_extracted_f792e8e8b92755a0433a3895ce0e35fc/usr/bin 
DEBUG: Doesn't match plugin regex, skipping: /tmp/appimage_extracted_f792e8e8b92755a0433a3895ce0e35fc/usr/bin/linuxdeploy 
DEBUG: output from plugin: 0
 
DEBUG: Found plugin 'appimage': /tmp/appimage_extracted_f792e8e8b92755a0433a3895ce0e35fc/usr/bin/linuxdeploy-plugin-appimage 
DEBUG: Doesn't match plugin regex, skipping: /tmp/appimage_extracted_f792e8e8b92755a0433a3895ce0e35fc/usr/bin/patchelf 
DEBUG: Doesn't match plugin regex, skipping: /tmp/appimage_extracted_f792e8e8b92755a0433a3895ce0e35fc/usr/bin/strip 
DEBUG: Searching for plugins in directory /home/brutus/.cache/briefcase/tools/linuxdeploy_plugins/qt 
terminate called after throwing an instance of 'std::logic_error'
  what():  subprocess failed (exit code 1)
>>> Return code: 127
Building...

Error while building app newapp.

Using qt in pyproject.toml: briefcase.2022_08_17-20_23_54.build.log

Using local Qt plugin (that's been patched): briefcase.2022_08_17-20_28_43.build.log

The error is only happening in Docker. I'd be curious if you're hitting the error as well with qt in pyproject.toml....if only to see whether its just my machine or not.

[freakboy3742]

I can confirm I'm seeing the same error in Docker; I'm wondering if this might be because of the ELF header - or possibly because the AppImage needs to be unpacked to run in Docker...

[rmartin16]

I also tried it unpatched and had the same problem.

This https://github.com/linuxdeploy/linuxdeploy/issues/154#issuecomment-932740990 does suggest unpacking may be a possible solution. Alternatively, there's even some interesting solutions around dd...

[freakboy3742]

I do so enjoy using a project where the project-recommended solution for getting something to run at all is "run dd over a binary"... (/me grumbles)

It would appear that the dd fix is at least partially overlapping with the patch-elf-binary fix... if the dd approach actually resolves the issue, then I guess we replace the elf patching with dd.

[rmartin16]

I was able to hack things together to get the unpacked Qt AppImage plugin to run in Docker....but that approach is going to encompass a lot more management for that style of plugin...

Initial testing with dd hasn't been successful.....I'll experiment some more tomorrow.

[rmartin16]

Ok....here's the story.

The dd command and our patch_elf_header() here are identical; they both just replace 3 bytes with zeros starting at the 8th byte of the file. The dd command didn't initially work for me because linuxdeploy was still trying to run the Qt plugin without extracting it first. To force all AppImages used in the container to be extracted first, I replaced the --appimage-extract-and-run for linuxdeploy with the environment variable APPIMAGE_EXTRACT_AND_RUN=1.

This all required a little bit of refactoring.....and I made the ELF file detection a bit more targeted.

[codecov[bot]]

Codecov Report

Merging #829 (11ce676) into main (ca1afb0) will increase coverage by 0.00%. The diff coverage is 100.00%.

Impacted Files	Coverage Δ
src/briefcase/integrations/linuxdeploy.py	100.00% <100.00%> (ø)
src/briefcase/platforms/linux/appimage.py	97.47% <100.00%> (+0.04%)	:arrow_up: