LibUSB from host OS + LibUdev from AppDir can lead to bad time (at least on Arch Linux)

[cculianu]

HI,

We use libusb for our project (https://github.com/Electron-Cash/Electron-Cash) to support various hardware devices via USB.

I was wondering why libusb is on the blacklist?

We end up copying it back to the appdir manually anyway right after it gets deleted by the blacklist code, because actually not having it in the appdir can cause the linker to pull it in from the user's system -- which creates potential linking issues and/or segfaults -- on Arch linux -- when libudev is pulled in from AppDir (but libusb was from host system).

I was wondering if you would consider removing it form the blacklist or if not -- can you tell me why it's blacklisted? From what we can gather it actually can run on any supported kernel version and.. like I said above -- blacklisting it risks having the user's version pulled in which creates more chaos than it solves.

I searched this repository and couldn't find the discussion or reason so I am curious.

Thanks in advance.

EDIT: It turns out we could really only reproduce this on Arch linux and -- usually it works fine in the setup described above. I am leaving this issue open in case others have libusb crash woes and want to find a workaround that works.

[cculianu]

Here is what one of the devs said, for background:

We (Electron-Cash devs) had a problem on some systems with crashing where the system libusb was loaded but this in turn loaded the libudev from the AppImage. This caused a crash in the dynamic linker relocation code. We have included libusb into our AppImage because of this. We also widely tested this AppImage on multiple distributions and found no issues. From what I can tell libusb should be usable independent of kernel version.

Electron-Cash/Electron-Cash@25d45fd

[probonopd]

I was wondering why libusb is on the blacklist?

The following are assumed to be part of the base system

We simply assumed that libusb is present in the default installation of all major distributions. Can you give an example of a major distribution where this is not the case?

[EchterAgo]

It's probably right that most distros have libusb, but it's problematic if Appimages include libudev and the system libusb depends on it. For me it's not about whether all systems have libusb but more about if there is a good reason not to include libusb, even if most systems have it.

[probonopd]

it's problematic if Appimages include libudev and the system libusb depends on it

So you are saying libudev and libusb need to be treated together, i.e., bundle both or bundle none? Do you have some evidence to back this up?

[cculianu]

@probonopd Yes -- that's exactly the issue. We'll get back to you tomorrow with a more detailed description and/or evidence (it's late-ish in our part of the world). But I felt the need to reply to affirm what you said. :)

[EchterAgo]

https://github.com/Electron-Cash/Electron-Cash/pull/1333#issuecomment-491124155

This is the original issue & what I found out about this. I'm still not quite sure about the root cause, might even be a bug in the linker, which is where the SIGSEGV was coming from. I will try to recreate this issue today.

[probonopd]

Thanks. Please try to reproduce on a distribution other than Arch Linux, which we think is unsupportable due to its "moving target" nature in which no two systems ever seem to be identical. Preferably use a Live ISO which means that we can reproduce the exact environment.

[cculianu]

Ah, so Arch is indeed a moving target, I see. Yes we did only actually see this in the field on Arch.

I am not sure we have the time to actually investigate this too deeply to make a case for adding it back -- the blacklist is advisory anyway as I understand it -- it's used as a way to remove libs that generally aren't needed, to save space, right? Devs like us are free to add them back if they are really needed, correct? (Sorry I'm a little new to AppImage).

We would like to investigate this more -- but it does appear that in some cases a user's app may depend on libusb (as ours did) which risks pulling in libusb from the host system which is linked against libudev -- which then pulls it in from AppDir first ... hence hilarity can ensue if they are compiled in an incompatible way (I know I am restating the issue again).

Since the blacklist is basically advisory-only to save space -- it's not crucial you guys remove it from the blacklist if you think it should remain in the blacklist.

In retrospect: We just.. wanted to get an idea of what your thoughts on it are as well as post the issue here in case others ran into this problem.

I renamed the issue to reflect what it really encapsulates.

We'll add more stuff here if we get time to figure out steps to reproduce outside of Arch linux... (it may just be an Arch-only strangeness/quirk...).

[probonopd]

Thanks for the additional context. If someone can show an AppImage that fails on a mainstream (non-Arch, non-Gentoo) desktop distribution due to this issue, we will definitely consider to remove libusb from the excludelist.

[cculianu]

Sounds reasonable. Hopefully you’ll never have a reason to. 🤞