graal icon indicating copy to clipboard operation
graal copied to clipboard
verified publisher icon oracle

[GR-51123] Native image fails to load a DLL from a path that contains non-ASCII characters

Open petoncle opened this issue 2 years ago • 9 comments

Native image fails to load a DLL from a path that contains non-ASCII characters... or from a path that contains the ~N tilde notation.

Describe the issue

This is a Windows only issue.

I have an app that uses JNA. JNA initially loads a DLL (jnidispatch.dll) after extracting it into a temporary folder that looks like this: C:\Users\<username>\AppData\Local\Temp\jna--187459531\jna11031635164979162837.dll

If the <username> contains non-ASCII characters (e.g. José), or if, in addition to containing non-ASCII characters, the username is too long (Alejandría) and Windows uses the ~N tilde notation (C:\Users\Alejan~1\AppData\...), the GraalVM native image fails loading the DLL:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: C:\Users\Alejan~1\AppData\Local\Temp\jna--187459531\jna11031635164979162837.dll
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryAbsolute(NativeLibrarySupport.java:100)
        at [email protected]/java.lang.ClassLoader.loadLibrary(ClassLoader.java:114)
        at [email protected]/java.lang.Runtime.load0(Runtime.java:852)
        at [email protected]/java.lang.System.load(System.java:2021)
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1045)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:1015)
        at com.sun.jna.Native.<clinit>(Native.java:221)
        at mousemaster.ExtendedKernel32.<clinit>(ExtendedKernel32.java:7)

Steps to reproduce the issue

To reproduce that problem, I just had to give JNA (-Djna.tmpdir) a path containing the ~N tilde notation.

  1. mkdir C:\Alejandría
  2. cd C:\Alejandría
  3. git clone https://github.com/petoncle/mousemaster .
  4. mvnw clean package
  5. mvnw -Pnative -Dagent package
  6. mv target\mousemaster.exe .
  7. mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\Alejan~1

Describe GraalVM and your environment:

  • GraalVM version: graalvm-jdk-21.0.1+12.1
  • JDK major version: 21
  • OS: Windows 10 and 11
  • Architecture: x86-64

More details

This works (non-native version running with graalvm-jdk-21.0.1+12.1):

C:\Alejandría>C:\graalvm-jdk-21.0.1+12.1\bin\java -Djna.debug_load=true -Djna.tmpdir=C:\Alejan~1 -jar mousemaster.jar
INFO: Found library resource at jar:file:/C:/Alejandr%c3%ada/mousemaster.jar!/com/sun/jna/win32-x86-64/jnidispatch.dll
Dec 26, 2023 4:23:13 PM com.sun.jna.Native extractFromResourcePath
INFO: Extracting library to C:\Alejan~1\jna2218230710365587904.dll
Dec 26, 2023 4:23:13 PM com.sun.jna.NativeLibrary loadLibrary
(...)

This does not work (native image):

C:\Alejandría>mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\Alejan~1
INFO: Found library resource at resource:/com/sun/jna/win32-x86-64/jnidispatch.dll
Dec 26, 2023 4:26:17 PM com.sun.jna.Native extractFromResourcePath
INFO: Extracting library to C:\Alejan~1\jna18325864350615041260.dll
Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: C:\Alejan~1\jna18325864350615041260.dll
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryAbsolute(NativeLibrarySupport.java:100)
(...)

This works (native image still, ~N tilde notation still, but only ASCII characters: Alejandro):

C:\Alejandro>mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\Alejan~1
INFO: Found library resource at resource:/com/sun/jna/win32-x86-64/jnidispatch.dll
Dec 26, 2023 4:33:13 PM com.sun.jna.Native extractFromResourcePath
INFO: Extracting library to C:\Alejan~1\jna10184328519890257310.dll
Dec 26, 2023 4:33:13 PM com.sun.jna.NativeLibrary loadLibrary
(...)

petoncle avatar Dec 26 '23 15:12 petoncle

Thanks For reporting, we are tracking it internally.

hamzaGhaissi avatar Dec 29 '23 11:12 hamzaGhaissi

Any updates on this? this is a serious problem

antonwiens avatar May 22 '25 12:05 antonwiens

As a temporary workaround, you can set -J-Dsun.jnu.encoding=Cp1252 during native image build.

antonwiens avatar May 22 '25 14:05 antonwiens

Thanks for looking into this @antonwiens. I've tried to add -J-Dsun.jnu.encoding=Cp1252 to my native-image.properties:

Args=-H:+AddAllCharsets -march=compatibility -J-Dsun.jnu.encoding=Cp1252

then built the native image:

JAVA_HOME=~/Documents/tools/graalvm-jdk-21.0.1+12.1/ ./mvnw -Pnative -Dagent package

but the DLL still could not be loaded:

c:\Alejandría>mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\Alejandría
2025-05-22T11:13:23.390 [main] INFO  com.sun.jna.Native - Looking in classpath from jdk.internal.loader.ClassLoaders$AppClassLoader@34ce8af7 for /com/sun/jna/win32-x86-64/jnidispatch.dll
2025-05-22T11:13:23.390 [main] INFO  com.sun.jna.Native - Found library resource at resource:/com/sun/jna/win32-x86-64/jnidispatch.dll
2025-05-22T11:13:23.397 [main] INFO  com.sun.jna.Native - Extracting library to C:\Alejandr�a\jna16178858372183860130.dll
Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: C:\Alejandr�a\jna16178858372183860130.dll
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryAbsolute(NativeLibrarySupport.java:100)
        at [email protected]/java.lang.ClassLoader.loadLibrary(ClassLoader.java:114)
        at [email protected]/java.lang.Runtime.load0(Runtime.java:852)
        at [email protected]/java.lang.System.load(System.java:2021)
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1045)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:1015)
        at com.sun.jna.Native.<clinit>(Native.java:221)
        at mousemaster.ExtendedKernel32.<clinit>(ExtendedKernel32.java:8)
        at mousemaster.WindowsClock.<clinit>(WindowsClock.java:10)
        at mousemaster.WindowsPlatform.<init>(WindowsPlatform.java:22)
        at mousemaster.MousemasterApplication.platform(MousemasterApplication.java:139)
        at mousemaster.MousemasterApplication.main(MousemasterApplication.java:104)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

Is there anything I'm doing wrong here?

petoncle avatar May 22 '25 15:05 petoncle

Sorry, i forgot to mention that you also need file encoding and native encoding -J-Dfile.encoding=Cp1252 -J-Dnative.encoding=Cp1252 -J-Dsun.jnu.encoding=Cp1252

Just tested it to be sure. At least it works for me with umlauts like öäü.

antonwiens avatar May 23 '25 07:05 antonwiens

Awesome, that works for me too (even though the folder name shows up incorrectly when printed in the console: C:\Alejandrφa instead of C:\Alejandría):

Args=-H:+AddAllCharsets -march=compatibility -J-Dfile.encoding=Cp1252 -J-Dnative.encoding=Cp1252 -J-Dsun.jnu.encoding=Cp1252

C:\Alejandría>mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\Alejandría
2025-05-23T05:42:18.318 [main] INFO  com.sun.jna.Native - Looking in classpath from jdk.internal.loader.ClassLoaders$AppClassLoader@34ce8af7 for /com/sun/jna/win32-x86-64/jnidispatch.dll
2025-05-23T05:42:18.318 [main] INFO  com.sun.jna.Native - Found library resource at resource:/com/sun/jna/win32-x86-64/jnidispatch.dll
### Not sure why the next log shows Alejandrφa instead of Alejandría. 
### But it's able to load the DLL anyway
### and the DLL is in C:\Alejandría (not C:\Alejandrφa).
2025-05-23T05:42:18.324 [main] INFO  com.sun.jna.Native - Extracting library to C:\Alejandrφa\jna615909175248744833.dll

I wonder why it's not working when I change Cp1252 to UTF-8:

Args=-H:+AddAllCharsets -march=compatibility -J-Dfile.encoding=UTF-8 -J-Dnative.encoding=UTF-8 -J-Dsun.jnu.encoding=UTF-8

C:\Alejandría>mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\Alejandría
2025-05-23T05:36:48.200 [main] INFO  com.sun.jna.Native - Looking in classpath from jdk.internal.loader.ClassLoaders$AppClassLoader@34ce8af7 for /com/sun/jna/win32-x86-64/jnidispatch.dll
2025-05-23T05:36:48.200 [main] INFO  com.sun.jna.Native - Found library resource at resource:/com/sun/jna/win32-x86-64/jnidispatch.dll
2025-05-23T05:36:48.207 [main] INFO  com.sun.jna.Native - Extracting library to C:\Alejandr�a\jna14709197415953247159.dll
Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: C:\Alejandr�a\jna14709197415953247159.dll
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryAbsolute(NativeLibrarySupport.java:100)

(I would have liked to make it work with non-Latin characters like C:\こんにちは.)

petoncle avatar May 23 '25 09:05 petoncle

I also noticed that the console output is always wrong, with utf-8 or cp1252. Did you try it with non-latin characters and it doesnt work?

Btw. setting to utf-8 doesnt work because i think that is the default

antonwiens avatar May 23 '25 09:05 antonwiens

I also noticed that the console output is always wrong, with utf-8 or cp1252. Did you try it with non-latin characters and it doesnt work?

This is what I get with non-latin characters:

C:\こんにちは>mousemaster.exe -Djna.debug_load=true -Djna.tmpdir=C:\こんにちは
2025-05-23T06:01:43.042 [main] WARN  com.sun.jna.Native - JNA Warning: IOException removing temporary files
java.io.IOException: JNA temporary directory 'C:\?????' does not exist
        at com.sun.jna.Native.getTempDir(Native.java:1357)
        at com.sun.jna.Native.removeTemporaryFiles(Native.java:1367)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:946)
        at com.sun.jna.Native.<clinit>(Native.java:221)
        at mousemaster.ExtendedKernel32.<clinit>(ExtendedKernel32.java:8)
        at mousemaster.WindowsClock.<clinit>(WindowsClock.java:10)
        at mousemaster.WindowsPlatform.<init>(WindowsPlatform.java:22)
        at mousemaster.MousemasterApplication.platform(MousemasterApplication.java:139)
        at mousemaster.MousemasterApplication.main(MousemasterApplication.java:104)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
2025-05-23T06:01:43.043 [main] INFO  com.sun.jna.Native - Looking in classpath from jdk.internal.loader.ClassLoaders$AppClassLoader@34ce8af7 for /com/sun/jna/win32-x86-64/jnidispatch.dll
2025-05-23T06:01:43.045 [main] INFO  com.sun.jna.Native - Found library resource at resource:/com/sun/jna/win32-x86-64/jnidispatch.dll
Exception in thread "main" java.lang.UnsatisfiedLinkError: Failed to create temporary file for /com/sun/jna/win32-x86-64/jnidispatch.dll library: JNA temporary directory 'C:\?????' does not exist
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1059)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:1015)
        at com.sun.jna.Native.<clinit>(Native.java:221)
        at mousemaster.ExtendedKernel32.<clinit>(ExtendedKernel32.java:8)
        at mousemaster.WindowsClock.<clinit>(WindowsClock.java:10)
        at mousemaster.WindowsPlatform.<init>(WindowsPlatform.java:22)
        at mousemaster.MousemasterApplication.platform(MousemasterApplication.java:139)
        at mousemaster.MousemasterApplication.main(MousemasterApplication.java:104)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

Btw. setting to utf-8 doesnt work because this is the default

Oh right that makes sense.

petoncle avatar May 23 '25 10:05 petoncle

However change file.encoding may have impact to application logic. If not handled properly, some strange characters will be stored.

It will be ideal if GraalVM can handle it without touch those properties.

crmky avatar May 31 '25 10:05 crmky