Importing wand takes 700ms on Ubuntu (and can't be avoided via environment)

[craigds]

In Ubuntu 18.04 LTS, the libmagickwand-6.q16-3 package installs the wand libraries at:

• /usr/lib/x86_64-linux-gnu/libMagickCore-6.Q16.so.3
• /usr/lib/x86_64-linux-gnu/libMagickWand-6.Q16.so.3

import wand finds these, but it takes ~700ms to do because it runs ctypes.find_library (apparently several times?)

Looking at the source code, it seemed like we could avoid this by setting MAGICK_HOME and WAND_MAGICK_LIBRARY_SUFFIX in our containers.

However, this doesn't actually help because wand hardcodes the .so extension, and .so.3 doesn't match.

Perhaps wand should handle a couple of environment variables containing the full paths to the libraries?

[emcconville]

I'm open to suggestions. Try NOT setting MAGICK_HOME, but just WAND_MAGICK_LIBRARY_SUFFIX=-6.Q16 value.

[craigds]

While checking that I found it interesting to note that the options doesn't seem to be required in our case. If I omit both environment variables and let wand run find_library over all the various combinations I get this:

>>> for suffix in suffixes:
...     libwand = find_library('MagickWand' + suffix)
...     print(f"{suffix:13s} : {libwand}")
...
...
...
              : None
HDRI          : None
HDRI-2        : None
-7            : None
-7HDRI        : None
-7HDRI-2      : None
-7.Q8         : None
-7.Q8HDRI     : None
-7.Q8HDRI-2   : None
-7.Q16        : None
-7.Q16HDRI    : None
-7.Q16HDRI-2  : None
-6            : libMagickWand-6.Q16.so.3
-6HDRI        : None
-6HDRI-2      : None
-Q16          : None
-Q16HDRI      : None
-Q16HDRI-2    : None
-Q8           : None
-Q8HDRI       : None
-Q8HDRI-2     : None
-6.Q16        : libMagickWand-6.Q16.so.3
-6.Q16HDRI    : None
-6.Q16HDRI-2  : None

It was surprising to me that the library is found while checking the -6 suffix even though it has the Q16 option tacked on to the library name. Perhaps the options are unnecessary? I wouldn't presume to know whether it is necessary on other systems though.

So the issue for us is that find_library is called 13 times before it finds a result. As you suggested, switching to setting the suffix but not MAGICK_HOME would solve this for us - find_library would run once and find the library immediately.

[emcconville]

It was surprising to me that the library is found while checking the -6 suffix even though it has the Q16 option tacked on to the library name.

That's ABI working at its best.

So the issue for us is that find_library is called 13 times before it finds a result.

I good time to consider migrating to ImageMagick-7. If performance is an important feature, I highly recommend ImageMagick-7's Q8 without HDRI. I know it's crummy to hear that, but Wand must exhaust ImageMagick-7 & support for High Dynamic Range images before attempting ImageMagick-6.

All that said, I still believe a "short-cut" environment variable is a good idea.

[craigds]

Imagemagick 7.x hasn't made it into ~an ubuntu LTS~ any ubuntu releases yet. Not sure why, seems like it's been around for years (?)

Looks like progress is being made though so maybe 22.04 will get it.