nut icon indicating copy to clipboard operation
nut copied to clipboard

Libraries not found (Ubuntu 24.04 Server arm)

Open sccfit opened this issue 1 year ago • 16 comments

Hi all... just tried to install nut on fresh install Ubuntu 24 LTS on raspberry pi, installed fine but running sudo nut-scanner -C results in following:

Cannot load USB library (libusb-1.0.so) : file not found. USB search disabled.
Cannot load SNMP library (libnetsnmp.so) : file not found. SNMP search disabled.
Cannot load XML library (libneon.so) : file not found. XML search disabled.
Cannot load AVAHI library (libavahi-client.so) : file not found. AVAHI search disabled.
Cannot load IPMI library (libfreeipmi.so) : file not found. IPMI search disabled.
Cannot load NUT library (libupsclient.so) : file not found. NUT search disabled.

I switched to Ubuntu because the package in it is closer to current than RPI OS, and I need the tools to connect to identical UPS on one device. I've confirmed libusb-1.0 is installed, but for some reason nut can't find it, do I need to create extra symlinks or something like that?

sccfit avatar May 03 '24 11:05 sccfit

Please retry with higher debug verbosity, e.g.:

:; sudo nut-scanner -DDDDDD -C > /tmp/nutscan.log 2>&1

There would be a huge wall of text about directories it looks into and file names it sees and tries to match there.

I wonder if your system delivers e.g. libusb-1.0.so.1.27.0 and no symlink like libusb-1.0.so which could be in another package.

For context: most of NUT deliverables can be split into packages based on dependencies (e.g. snmp, ipmi, usb etc. drivers) or purpose (client, cgi, ...) However the nut-scanner tool can optionally talk all protocols it can discover, and with usual shared object linking approach it would require all those libraries installed - even if the particular user never needs them (only got USB and has no care and footprint to spare for SNMP or IPMI libs). Instead, shared objects (libraries) are loaded dynamically by the program itself at its discretion - or skipped if the load failed, such as in this case. Why specifically it failed is the root cause to discover here.

jimklimov avatar May 03 '24 11:05 jimklimov

Also, just in case: how did you install NUT (packages? custom build?) and which version?

jimklimov avatar May 03 '24 12:05 jimklimov

Must be the OS that NUT was compiled on, I’ve just compiled and installed on Ubuntu 24.04 no issues.

danmrossi avatar May 07 '24 10:05 danmrossi

Sorry, have been on holidays, looping back to this now. I'm not great with Linux/packages/github, I installed using apt which I believe should be version 2.8.1-3.1ubuntu2.

Have run with higher debug verbosity, the following seem to be the most pertinent lines:

   0.001188     [D1]    Built-in:       /usr/lib/gcc/aarch64-unknown-linux-gnu
   0.001239     [D2] Looking for lib libusb-1.0.so in directory #0 : /usr/lib/aarch64-linux-gnu

   -- snip --
   
   0.012290     [D5] Comparing lib libusb-1.0.so with dirpath entry libusb-1.0.so.0
   0.012298     [D5] Comparing lib libusb-1.0.so with dirpath entry libusb-1.0.so.0.4.0

   -- snip --
   
   0.014574     [D1] Looking for lib libusb-1.0.so, found NULL
   
   -- snip --
   0.026829     [D1] nutscan_init: get_libname() did not resolve libname for LibUSB, trying to load it with libtool default resolver
   Cannot load USB library (libusb-1.0.so) : file not found. USB search disabled.

Am I right in thinking that creating a symlink 'libusb-1.0.so' that points to 'libusb-1.0.so.0' in the same directory should resolve the issue in this case?

sccfit avatar May 17 '24 00:05 sccfit

Yes, I wonder why one was not created in the first place (maybe devel packages deliver such a link, or some distro conventions had changed?) :\

At least, not directly a NUT problem, or not sure quickly how to address such distro specifics in common codebase, short of adding a nut-scanner config file/options about such paths.

jimklimov avatar May 17 '24 07:05 jimklimov

Hm, yeah, the name without trailing numbers seems to come from development packages:

:; grep libusb-1.0.so /var/lib/dpkg/info/*.list
/var/lib/dpkg/info/libusb-1.0-0-dev:amd64.list:/usr/lib/x86_64-linux-gnu/libusb-1.0.so
/var/lib/dpkg/info/libusb-1.0-0:amd64.list:/usr/lib/x86_64-linux-gnu/libusb-1.0.so.0.3.0
/var/lib/dpkg/info/libusb-1.0-0:amd64.list:/usr/lib/x86_64-linux-gnu/libusb-1.0.so.0

Wondering how nobody stumbled on that before. Did everyone have a larger system footprint installed (e.g. did other packagings of NUT depend on *-dev packages of other stuff)?..

At least, this does seem like something NUT codebase can address - beside the idea above for direct run-time modifiable configuration of file names/paths to use, it may be possible to add the name(s) of the library seen at configuration/build time (e.g. libusb-1.0.so.0 or resolved through all the symlinks to "real file" at libusb-1.0.so.0.3.0) to the built-in search list.

I would be wary of just allowing random suffixes to be accepted in the search method; to that end, get_libname_in_dir() explicitly has checks to avoid "*.dll.a", ".so.1.2.3" etc. and only pick the specified file name exactly (notably, not all such names can even be shared objects that we would be able to dynamically load). Although that is still a clumsy option... "If we found nothing exactly, try again with random suffixes allowed and check by dlload()" but that IMHO is an attack waiting to happen :)

jimklimov avatar May 17 '24 08:05 jimklimov

A practical search on a test system does use realpath() (where available) and practically use the ultimate file name when it looks at a symlink (which does exist):

   0.002097     [D5] Comparing lib libusb-1.0.so with dirpath entry libusb-1.0.so
   0.002108     [D2] Candidate path for lib libusb-1.0.so is /usr/lib/x86_64-linux-gnu/libusb-1.0.so (realpath /usr/lib/x86_64-linux-gnu/libusb-1.0.so.0.3.0)
   0.002119     [D1] Looking for lib libusb-1.0.so, found /usr/lib/x86_64-linux-gnu/libusb-1.0.so.0.3.0
   0.002121     [D1] nutscan_init: get_libname() resolved '/usr/lib/x86_64-linux-gnu/libusb-1.0.so.0.3.0' for LibUSB, loading it
   0.002324     [D1] nutscan_init: succeeded to load the library for LibUSB

Looking at code, I see that for each library of interest, nut-scanner tries one or more patterns via get_libname() and if any of those fails to find an exact filename match in searched directories, it falls back to the likes of nutscan_load_usb_library() (half a dozen of those for different libs) which would try to lt_dlopen() the specified pattern(s) in hopes that the OS resolver would take care of locating the library in some location unexpected by NUT (and maybe under a slightly different name - up to the OS not NUT at this point). This is the part seen in your log too, as:

   0.026829     [D1] nutscan_init: get_libname() did not resolve libname for LibUSB, trying to load it with libtool default resolver

jimklimov avatar May 17 '24 13:05 jimklimov

Running some experiments with another lib that is easier to fiddle with (NUT only built in this system, not installed into common paths, so its own libupsclient is not resolved via OS paths):

:; ./tools/nut-scanner/nut-scanner -O
Cannot load NUT library (libupsclient.so) : file not found. NUT search disabled.

With the local build in the library path, it is seen:

:; LD_LIBRARY_PATH="`pwd`/clients/.libs:$LD_LIBRARY_PATH" ./tools/nut-scanner/nut-scanner -O -DDDDDD 2>&1  | grep -B5 -A5 'libupsclient\.so\.'

...
   0.014299     [D5] Comparing lib libupsclient.so with dirpath entry libupsclient.so
   0.014308     [D2] Candidate path for lib libupsclient.so is /home/jim/nut/clients/.libs/libupsclient.so (realpath /home/jim/nut/clients/.libs/libupsclient.so.6.0.1)
   0.014312     [D2] Looking for lib libupsclient.so, found in LD_LIBRARY_PATH
   0.014313     [D1] Looking for lib libupsclient.so, found /home/jim/nut/clients/.libs/libupsclient.so.6.0.1
   0.014313     [D1] nutscan_init: get_libname() resolved '/home/jim/nut/clients/.libs/libupsclient.so.6.0.1' for NUT Client library, loading it
   0.014361     [D1] nutscan_init: succeeded to load the library for NUT Client library
...

Removing the .so symlink out of the way:

:; ls -la clients/.libs/libupsclient.so*
lrwxrwxrwx 1 jim jim     21 May 15 17:35 clients/.libs/libupsclient.so -> libupsclient.so.6.0.1
lrwxrwxrwx 1 jim jim     21 May 15 17:35 clients/.libs/libupsclient.so.6 -> libupsclient.so.6.0.1
-rwxr-xr-x 1 jim jim 223376 May 15 17:35 clients/.libs/libupsclient.so.6.0.1

:; mv clients/.libs/libupsclient.so{,.x}

:; LD_LIBRARY_PATH="`pwd`/clients/.libs:$LD_LIBRARY_PATH" ./tools/nut-scanner/nut-scanner -O -DDDDDD 2>&1  | grep -B5 -A5 'libupsclient\.so\.'

...
   0.013544     [D2] Looking for lib libupsclient.so in directory #1 : /home/jim/nut/clients/.libs
   0.013569     [D5] Comparing lib libupsclient.so with dirpath entry libnutclient.so.2.0.2
   0.013586     [D5] Comparing lib libupsclient.so with dirpath entry libupsclient.a
   0.013588     [D5] Comparing lib libupsclient.so with dirpath entry upslog
   0.013589     [D5] Comparing lib libupsclient.so with dirpath entry libnutclientstub.la
   0.013590     [D5] Comparing lib libupsclient.so with dirpath entry libupsclient.so.6.0.1
   0.013591     [D5] Comparing lib libupsclient.so with dirpath entry libupsclient.exp
...
   0.013598     [D5] Comparing lib libupsclient.so with dirpath entry libupsclient.so.x
   0.013599     [D5] Comparing lib libupsclient.so with dirpath entry libnutclient.a
   0.013600     [D5] Comparing lib libupsclient.so with dirpath entry libnutclientstub.a
   0.013600     [D5] Comparing lib libupsclient.so with dirpath entry upsset.cgi
   0.013601     [D5] Comparing lib libupsclient.so with dirpath entry libupsclient.so.6
...
   0.013613     [D1] Got no strong candidate path for lib libupsclient.so in /home/jim/nut/clients/.libs, but saw seemingly related names (are you missing a symbolic link, perhaps?) e.g.: libupsclient.so.6.0.1
   0.013619     [D2] Looking for lib libupsclient.so in directory #2 : /usr/lib/x86_64-linux-gnu
...

So it even recognizes that file, just not as a "strong" candidate.

jimklimov avatar May 17 '24 13:05 jimklimov

As a side note, this may also be a concern for DMF effort (currently awaiting its destiny in another branch):

  • https://app.circleci.com/pipelines/github/jimklimov/nut/1663/workflows/39cda40a-9cdd-437f-bc47-8ac693f290e0/jobs/4951 after PR #2275 for issue track #1316
...
Error loading Neon library required for DMF: libneon.so not found by dynamic loader; please verify it is in your /usr/lib or some otherwise searched dynamic-library path, under this exact name (maybe you just need a symlink?)
Error loading Neon library required for DMF: libneon-gnutls.so not found by dynamic loader; please verify it is in your /usr/lib or some otherwise searched dynamic-library path, under this exact name (maybe you just need a symlink?)
=== DMF-Test: Error parsing data: -1

jimklimov avatar Jun 15 '24 15:06 jimklimov

Same issue here, with Alpine Linux: https://gitlab.alpinelinux.org/alpine/aports/-/issues/16216

The libusb package only provides:

/usr/lib/libusb-1.0.so.0
/usr/lib/libusb-1.0.so.0.4.0

The libusb-dev package provides:

/usr/include/libusb-1.0/libusb.h
/usr/lib/libusb-1.0.a
/usr/lib/libusb-1.0.so
/usr/lib/pkgconfig/libusb-1.0.pc

Maybe a regression of this old issue https://github.com/networkupstools/nut/issues/233

alexojegu avatar Jun 18 '24 17:06 alexojegu

Thanks for confirming this is more wide-spread ;( and also on the radar for so long...

jimklimov avatar Jun 18 '24 23:06 jimklimov

This patch seems to work:

--- a/common/common.c
+++ b/common/common.c
@@ -3282,6 +3282,8 @@ static char * get_libname_in_dir(const char* base_libname, size_t base_libname_l
 
 	if (libname_alias) {
 		if (!libname_path) {
+			if (libname_alias[base_libname_length] == '.')
+				return libname_alias;
 			upsdebugx(1, "Got no strong candidate path for lib %s in %s"
 				", but saw seemingly related names (are you missing"
 				" a symbolic link, perhaps?) e.g.: %s",

I am not certain if it is robust though.

sertonix avatar Jun 22 '24 12:06 sertonix

My concern would be about picking a somewhat random library this way. Say libusb.so.(0|1) on some system meant the two different ABIs of separate generations of the library. Linking would be catastrophic!

I'm more in favor of:

  • adding envvars to specify a pathname to specific library file;
  • doing some tricks during configure to find the exact library name a standard dynamic link would use, and use that name (if known) preferentially. Not sure it can be done portably, but for many systems can be done at least.

jimklimov avatar Jun 22 '24 17:06 jimklimov

Well, that is kind of already the case since there is no guarantee where the libusb.so symlink leads to. Ideally the sonames of the libraries nut was compiled against would be baked in the binary. No idea how to do that though.

sertonix avatar Jun 22 '24 17:06 sertonix

Seems to work, even on a Mac:

:; LD_LIBRARY_PATH="`pwd`/clients/.libs" ./tools/nut-scanner/nut-scanner -D
...
   0.000915     [D1] Looking for lib libusb-1.0.0.dylib, found /usr/local/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib
   0.000933     [D1] nutscan_init: get_libname() resolved '/usr/local/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' for LibUSB, loading it
   0.006119     [D1] nutscan_init: succeeded to load the library for LibUSB
   0.006129     [D1] nutscan_init: skipped loading the library for LibSNMP: was absent during NUT build
   0.006401     [D1] Looking for lib libneon.27.dylib, found /usr/local/Cellar/neon/0.33.0/lib/libneon.27.dylib
   0.006409     [D1] nutscan_init: get_libname() resolved '/usr/local/Cellar/neon/0.33.0/lib/libneon.27.dylib' for LibNeon, loading it
   0.007187     [D1] nutscan_init: succeeded to load the library for LibNeon
   0.007196     [D1] nutscan_init: skipped loading the library for LibAvahi: was absent during NUT build
   0.007451     [D1] Looking for lib libfreeipmi.17.dylib, found /usr/local/Cellar/freeipmi/1.6.14/lib/libfreeipmi.17.dylib
   0.007459     [D1] nutscan_init: get_libname() resolved '/usr/local/Cellar/freeipmi/1.6.14/lib/libfreeipmi.17.dylib' for LibFreeIPMI, loading it
   0.010409     [D1] nutscan_init: succeeded to load the library for LibFreeIPMI
   0.010499     [D1] Looking for lib libupsclient.dylib, found /Users/abuild/nut/clients/.libs/libupsclient.6.dylib
   0.010506     [D1] nutscan_init: get_libname() resolved '/Users/abuild/nut/clients/.libs/libupsclient.6.dylib' for NUT Client library, loading it
   0.011017     [D1] nutscan_init: succeeded to load the library for NUT Client library
...

jimklimov avatar Jul 03 '24 23:07 jimklimov

Waiting for a release :smile:

alexojegu avatar Jul 05 '24 11:07 alexojegu

Waiting for a release 😄

Me too, wondering when it's going to be released :) Is it possible to install it on ubuntu with sudo apt install nut or do I need to compile it my self?

rwidmark avatar Jul 09 '24 12:07 rwidmark

Currently we do not make snapshot packages (the idea is in backlog though), so your best shot for the latest and greatest features (and mishaps) is https://github.com/networkupstools/nut/wiki/Building-NUT-for-in%E2%80%90place-upgrades-or-non%E2%80%90disruptive-tests

jimklimov avatar Jul 09 '24 19:07 jimklimov

With #2532 in place, it seems all reasonable goals of this issue have been achieved.

jimklimov avatar Jul 16 '24 15:07 jimklimov

So what was the final verdict on a fix for this? I just tried installing nut on a new Mint 22 install and got the same error. Is the above listed commit the fix itself, and the version including the fix just isn't included in Mint's apt repo? If that's the case, does nut happen to have a ppa that can be enabled for installing newer versions, or is compiling the only option?

Majoraslayer avatar Aug 05 '24 00:08 Majoraslayer

Currently, to get recent changes (master branch; the recent change above is not part of any numbered release yet) custom-compiling is the only official option; the https://github.com/networkupstools/nut/wiki/Building-NUT-for-in%E2%80%90place-upgrades-or-non%E2%80%90disruptive-tests page (and docs it references) should help navigate through this.

Another matter is that Debian-like OSes share the recipes and many of those are several NUT releases behind the actual publications. So if your distro serves something older than 2.8.2 (still without this fix, but generally speaking), for current NUT you might be better served by their "experimental" repos or custom builds as well.

There is a backlogged effort to import several packaging recipes (some pioneered in FTY fork and branch, queued per #1316) and to publish the resulting snapshot packages whether with our NUT CI farm or using the OBS per #1209. The plan seems clear, but it just was not yet addressed. Too many efforts, too few people :-\

jimklimov avatar Aug 05 '24 08:08 jimklimov

Any updates on this. Just installed today on Ubuntu and got the same issue:

Linux data 6.8.0-49-generic #49-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 4 02:06:24 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

"Ubuntu 24.04.1 LTS" VERSION_ID="24.04" VERSION="24.04.1 LTS (Noble Numbat)" VERSION_CODENAME=noble

nut-scanner Cannot load USB library (libusb-1.0.so) : file not found. USB search disabled. Cannot load SNMP library (libnetsnmp.so) : file not found. SNMP search disabled. Cannot load XML library (libneon.so) : file not found. XML search disabled. Cannot load AVAHI library (libavahi-client.so) : file not found. AVAHI search disabled. Cannot load IPMI library (libfreeipmi.so) : file not found. IPMI search disabled. Cannot load NUT library (libupsclient.so) : file not found. NUT search disabled.

hibfamtv avatar Dec 06 '24 17:12 hibfamtv

See above.

jimklimov avatar Dec 06 '24 18:12 jimklimov

I guess I am also missing where the fix is for this. I didn't have problems a couple months ago when I installed this on a Pi but now that I am trying to do it on a Ubuntu installation I'm getting the same as the above, no matter how I install it.

Can someone in a bit more detail explain what needs to be done to get this to work? The article linked above is a bit vague about how to get this working and seems to be more a general piece about packaging this.

edwinbrett avatar Dec 29 '24 06:12 edwinbrett

As a quick hotfix, ensure the filenames that your build of nut-scanner looks for, like libXXX.so or libXXX.so.N, do exist. They may be symlinks delivered by libXXX-dev packages or can be made manually in same locations your libXXX.so.N.M.P actual library files reside, typically /usr/lib or /usr/lib64 or some complex path with architecture name; find tool is your friend here.

jimklimov avatar Dec 29 '24 06:12 jimklimov

This workaround enabled my nut-scanner to recognise the SNMP library.

ln -s /usr/lib64/libnetsnmp.so.40.1.0 /usr/lib64/libnetsnmp.so

edmondsiu0 avatar Jan 23 '25 15:01 edmondsiu0

Seeing the same thing on Ubuntu 24.04 server

$ sudo nut-scanner -U
Cannot load USB library (/usr/lib/x86_64-linux-gnu/libusb-0.1.so.4.4.4) : /usr/lib/x86_64-linux-gnu/libusb-0.1.so.4.4.4: undefined symbol: libusb_init. USB search disabled.
Cannot load SNMP library (libnetsnmp.so) : file not found. SNMP search disabled.
Cannot load XML library (libneon.so) : file not found. XML search disabled.
Cannot load AVAHI library (libavahi-client.so) : file not found. AVAHI search disabled.
Cannot load IPMI library (libfreeipmi.so) : file not found. IPMI search disabled.
Cannot load NUT library (libupsclient.so) : file not found. NUT search disabled.

And the file /usr/lib/libusb-1.0.so already exists. I don't fully understand how the symlink is supposed to work here

karthikrao5 avatar Feb 08 '25 13:02 karthikrao5

libusb-0.1.so.4.4.4: undefined symbol: libusb_init. USB search disabled.

This one looks odd - IIRC libusb* prefixed names are of libusb-1.0 API. Perhaps your package was built against one generation of the library and you installed it with another? Possibly also the libusb-0.1 you have is actually the "compat" project, recreating the 0.1 API mapped to 1.0 implementation (needs to be installed).

jimklimov avatar Feb 08 '25 14:02 jimklimov

appreciate this is closed, it might help anyone running ubuntu that wants usb scanning;

hub@hub:~$ cat /etc/debian_version 
trixie/sid
hub@hub:~$ sudo ln -s /usr/lib/x86_64-linux-gnu/libusb-1.0.so.0.4.0 /usr/lib/x86_64-linux-gnu/libusb.so
hub@hub:~$ sudo nut-scanner -C
Cannot load SNMP library (libnetsnmp.so) : file not found. SNMP search disabled.
Cannot load XML library (libneon.so) : file not found. XML search disabled.
Cannot load AVAHI library (libavahi-client.so) : file not found. AVAHI search disabled.
Cannot load IPMI library (libfreeipmi.so) : file not found. IPMI search disabled.
Cannot load NUT library (libupsclient.so) : file not found. NUT search disabled.
Scanning USB bus.
[nutdev1]
        driver = "usbhid-ups"
        port = "auto"
        vendorid = "0764"
        productid = "0501"
        product = "CP1500EPFCLCD"
        serial = "CTLLN2000041"
        vendor = "CPS"
        bus = "001"
        device = "004"
        busport = "006"
        ###NOTMATCHED-YET###bcdDevice = "0001"

you can change the name of the linked file to get any of the others

markwalkom avatar Feb 09 '25 09:02 markwalkom

Out of curiosity, is the problem reproducible with current NUT codebase (master branch, eventually to become 2.8.3 release when some other chores are done)?

jimklimov avatar Feb 09 '25 12:02 jimklimov