Ktor no netty_transport_native_epoll_x86_64 in java.library.path

[softshipper]

Hi all

During starting Ktor application, I've got the following logs:

java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1871)
    at java.lang.Runtime.loadLibrary0(Runtime.java:872)
    at java.lang.System.loadLibrary(System.java:1124)
    at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
    at java.security.AccessController.doPrivileged(Native Method)
    at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
    at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
    at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
    at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:231)
    at io.netty.channel.epoll.Native.<clinit>(Native.java:58)
    at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
    at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
    at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
    at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
22:56:07.760 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - netty_transport_native_epoll_x86_64 cannot be loaded from java.library.path, now trying export to -Dio.netty.native.workdir: /tmp
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1871)
    at java.lang.Runtime.loadLibrary0(Runtime.java:872)
    at java.lang.System.loadLibrary(System.java:1124)
    at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
    at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
    at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
    at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:231)
    at io.netty.channel.epoll.Native.<clinit>(Native.java:58)
    at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
    at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
    at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
    at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
    Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_x86_64 in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1871)
        at java.lang.Runtime.loadLibrary0(Runtime.java:872)
        at java.lang.System.loadLibrary(System.java:1124)
        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
        at java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
        ... 7 common frames omitted
22:56:07.761 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - Unable to load the library 'netty_transport_native_epoll', trying other loading mechanism.
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1871)
    at java.lang.Runtime.loadLibrary0(Runtime.java:872)
    at java.lang.System.loadLibrary(System.java:1124)
    at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
    at java.security.AccessController.doPrivileged(Native Method)
    at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
    at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
    at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
    at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:234)
    at io.netty.channel.epoll.Native.<clinit>(Native.java:58)
    at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
    at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
    at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
    at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
22:56:07.761 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - netty_transport_native_epoll cannot be loaded from java.library.path, now trying export to -Dio.netty.native.workdir: /tmp
java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1871)
    at java.lang.Runtime.loadLibrary0(Runtime.java:872)
    at java.lang.System.loadLibrary(System.java:1124)
    at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
    at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
    at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
    at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:234)
    at io.netty.channel.epoll.Native.<clinit>(Native.java:58)
    at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
    at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
    at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
    at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
    Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1871)
        at java.lang.Runtime.loadLibrary0(Runtime.java:872)
        at java.lang.System.loadLibrary(System.java:1124)
        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
        at java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
        ... 7 common frames omitted

the build file

plugins {
    application
    kotlin("jvm") version "1.3.61"
}

group = "io.oceanstack"
version = "1.0-SNAPSHOT"

val ktor_version = "1.3.1"

repositories {
    mavenCentral()
    jcenter()
}

dependencies {
    implementation(kotlin("stdlib-jdk8"))
    compile("io.ktor:ktor-server-core:$ktor_version")
    compile("io.ktor:ktor-server-netty:$ktor_version")
    compile("ch.qos.logback:logback-classic:1.2.3")
    testCompile(group = "junit", name = "junit", version = "4.12")
}

tasks {
    compileKotlin {
        kotlinOptions.jvmTarget = "1.8"
    }
    compileTestKotlin {
        kotlinOptions.jvmTarget = "1.8"
    }
}

application {
    mainClassName = "io.ktor.server.netty.EngineMain"
} 

What am I doing wrong?

The Java runtime:

java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (Zulu 8.44.0.11-CA-linux64) (build 1.8.0_242-b20)
OpenJDK 64-Bit Server VM (Zulu 8.44.0.11-CA-linux64) (build 25.242-b20, mixed mode)

Thanks

[volkert-fastned]

@softshipper This issue has been raised before. I just documented a (Gradle-specific) workaround here, along with a request to have it automatically handled somehow if possible: https://github.com/ktorio/ktor/issues/1672#issue-569857300

Note that although the workaround will allow the native transport to successfully load on startup (if available for the underlying platform), it will unfortunately not eliminate all the exceptions that Netty throws at the debug level before it ultimately succeeds in loading the native transport. The reason for this being that it tries multiple ways of locating and loading the native transport library, the first ones of which always fail (at least in the case of Ktor).

See also this answer at the Netty GitHub project: https://github.com/netty/netty/issues/8263#issuecomment-418696569

[softshipper]

@volkert-fastned There is no way to prevent the error message right?

[volkert-fastned]

@softshipper As I understand it, not without increasing the logging level for the specific Netty package from DEBUG to INFO. How to do this, depends on what logging backend you're using in your project.

If you're using Log4j 2 as your logging backend, you can add a package-specific override as follows:

        <AsyncLogger name="io.netty.util.internal.NativeLibraryLoader" level="info" includeLocation="true" additivity="false">
            <AppenderRef ref="[INSERT_APPENDER_NAME_HERE]"/>
        </AsyncLogger>

Be sure to use a regular Logger here instead of an AsyncLogger, if you're not using asynchronous logging.

Of course this will also suppress the "successfully loaded" message. But if you want a confirmation in your log that the native transport has successfully been loaded and is in use, I believe you can actually query Netty programmatically for that and then log something in your code.

[oleg-larshin]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.