Native compilation not possible when using com.azure dependency - io.netty conflict

[Cobas91]

Describe the bug

Hey guys,

i have installed these two dependency on my project.

<dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>msal4j</artifactId>
            <version>1.12.0</version>
        </dependency>
<dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-identity</artifactId>
            <version>1.5.2</version>
</dependency>

When i try to compile it nativly i got this error:

Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of io.netty.handler.ssl.OpenSslClientContext are allowed in the image heap as this class should be initialized at image runtime. To see how this object got instantiated use --trace-object-instantiation=io.netty.handler.ssl.OpenSslClientContext.
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.checkImageHeapInstance(ClassInitializationFeature.java:135)
        at com.oracle.graal.pointsto.meta.AnalysisUniverse.replaceObject(AnalysisUniverse.java:582)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.replaceObject(AnalysisConstantReflectionProvider.java:257)
        at com.oracle.svm.hosted.ameta.AnalysisConstantReflectionProvider.interceptValue(AnalysisConstantReflectionProvider.java:228)
        at com.oracle.svm.hosted.heap.SVMImageHeapScanner.transformFieldValue(SVMImageHeapScanner.java:126)
        at com.oracle.graal.pointsto.heap.ImageHeapScanner.onFieldValueReachable(ImageHeapScanner.java:331)
        at com.oracle.graal.pointsto.heap.ImageHeapScanner.lambda$createImageHeapObject$3(ImageHeapScanner.java:272)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        ... 10 more

After some research i figured out that the azur dependencys uses the same io.netty dependency as quarkus.

i tried to instantiated the complete azure dependency on run time via: --initialize-at-run-time=com.azure

This leads in the following error:

Error: Classes that should be initialized at run time got initialized during image building:
 io.netty.buffer.UnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledUnsafeDirectByteBuf
io.netty.buffer.AbstractReferenceCountedByteBuf the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.AbstractReferenceCountedByteBuf'). To see why io.netty.buffer.AbstractReferenceCountedByteBuf got initialized use --trace-class-initialization=io.netty.buffer.AbstractReferenceCountedByteBuf
io.netty.buffer.UnpooledHeapByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledHeapByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledHeapByteBuf
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf
io.netty.buffer.AbstractPooledDerivedByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.AbstractPooledDerivedByteBuf got initialized use --trace-class-initialization=io.netty.buffer.AbstractPooledDerivedByteBuf
io.netty.buffer.PooledByteBufAllocator the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.PooledByteBufAllocator' and from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'PooledByteBufAllocator.class'). To see why io.netty.buffer.PooledByteBufAllocator got initialized use --trace-class-initialization=io.netty.buffer.PooledByteBufAllocator
io.netty.buffer.UnpooledDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledDirectByteBuf
io.netty.buffer.ByteBufUtil the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.ByteBufUtil' and from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'ByteBufUtil.class'). To see why io.netty.buffer.ByteBufUtil got initialized use --trace-class-initialization=io.netty.buffer.ByteBufUtil
io.netty.buffer.ByteBufAllocator the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.ByteBufAllocator' and from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'ByteBufAllocator.class'). To see why io.netty.buffer.ByteBufAllocator got initialized use --trace-class-initialization=io.netty.buffer.ByteBufAllocator
io.netty.buffer.PooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.PooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.PooledUnsafeDirectByteBuf
io.netty.buffer.PooledSlicedByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.PooledSlicedByteBuf got initialized use --trace-class-initialization=io.netty.buffer.PooledSlicedByteBuf
io.netty.handler.ssl.ReferenceCountedOpenSslContext the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'ReferenceCountedOpenSslContext.class'). To see why io.netty.handler.ssl.ReferenceCountedOpenSslContext got initialized use --trace-class-initialization=io.netty.handler.ssl.ReferenceCountedOpenSslContext
io.netty.buffer.PooledByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.PooledByteBuf got initialized use --trace-class-initialization=io.netty.buffer.PooledByteBuf
io.netty.util.AbstractReferenceCounted the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\common\native-image.properties' in 'null' with 'io.netty.util.AbstractReferenceCounted'). To see why io.netty.util.AbstractReferenceCounted got initialized use --trace-class-initialization=io.netty.util.AbstractReferenceCounted

com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
 io.netty.buffer.UnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledUnsafeDirectByteBuf
io.netty.buffer.AbstractReferenceCountedByteBuf the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.AbstractReferenceCountedByteBuf'). To see why io.netty.buffer.AbstractReferenceCountedByteBuf got initialized use --trace-class-initialization=io.netty.buffer.AbstractReferenceCountedByteBuf
io.netty.buffer.UnpooledHeapByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledHeapByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledHeapByteBuf
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf
io.netty.buffer.AbstractPooledDerivedByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.AbstractPooledDerivedByteBuf got initialized use --trace-class-initialization=io.netty.buffer.AbstractPooledDerivedByteBuf
io.netty.buffer.PooledByteBufAllocator the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.PooledByteBufAllocator' and from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'PooledByteBufAllocator.class'). To see why io.netty.buffer.PooledByteBufAllocator got initialized use --trace-class-initialization=io.netty.buffer.PooledByteBufAllocator
io.netty.buffer.UnpooledDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledDirectByteBuf
io.netty.buffer.ByteBufUtil the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.ByteBufUtil' and from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'ByteBufUtil.class'). To see why io.netty.buffer.ByteBufUtil got initialized use --trace-class-initialization=io.netty.buffer.ByteBufUtil
io.netty.buffer.ByteBufAllocator the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\buffer\native-image.properties' in 'null' with 'io.netty.buffer.ByteBufAllocator' and from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'ByteBufAllocator.class'). To see why io.netty.buffer.ByteBufAllocator got initialized use --trace-class-initialization=io.netty.buffer.ByteBufAllocator
io.netty.buffer.PooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.PooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.PooledUnsafeDirectByteBuf
io.netty.buffer.PooledSlicedByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.PooledSlicedByteBuf got initialized use --trace-class-initialization=io.netty.buffer.PooledSlicedByteBuf
io.netty.handler.ssl.ReferenceCountedOpenSslContext the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis with 'ReferenceCountedOpenSslContext.class'). To see why io.netty.handler.ssl.ReferenceCountedOpenSslContext got initialized use --trace-class-initialization=io.netty.handler.ssl.ReferenceCountedOpenSslContext
io.netty.buffer.PooledByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.PooledByteBuf got initialized use --trace-class-initialization=io.netty.buffer.PooledByteBuf
io.netty.util.AbstractReferenceCounted the class was requested to be initialized at run time (from 'META-INF\native-image\io.netty\common\native-image.properties' in 'null' with 'io.netty.util.AbstractReferenceCounted'). To see why io.netty.util.AbstractReferenceCounted got initialized use --trace-class-initialization=io.netty.util.AbstractReferenceCounted

        at com.oracle.svm.core.util.UserError.abort(UserError.java:72)
        at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:560)
        at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:167)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$10(NativeImageGenerator.java:726)
        at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:74)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$11(NativeImageGenerator.java:726)
        at com.oracle.graal.pointsto.PointsToAnalysis.runAnalysis(PointsToAnalysis.java:751)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:723)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:558)
        at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:515)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:407)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:585)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:615)

I guess the problem is that com.azure and quarkus are using the io.netty dependency

Looks for me like a bug

Is there a way to use the ---fallback option in specific dependencys. For me it would be okay if i have to put the azure.jar next to the native executable to run the application.

Expected behavior

When i declare that the com.azure libary should build at runtime the classes for quarkus should not be afffected

Actual behavior

No response

How to Reproduce?

I can give access to the private repository for selected ppl.

Output of uname -a or ver

No response

Output of java -version

openjdk version "17.0.3" 2022-04-19 OpenJDK Runtime Environment GraalVM CE 22.1.0 (build 17.0.3+7-jvmci-22.1-b06) OpenJDK 64-Bit Server VM GraalVM CE 22.1.0 (build 17.0.3+7-jvmci-22.1-b06, mixed mode, sharing)

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.10.2.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.4

Additional information

No response

[cescoffier]

My guess is the usage of reactor-netty that we do not support in native. We have implemented a more friendly HTTP client for the azure SDK (but I need to see how we can use it).

[Cobas91]

@cescoffier Your answer confused me a little bit. Is there something that i should do in this stage? If i can do something (get more informations from the build process etc.) ping me

[cescoffier]

It looks like one of the dependency you declare has a dependency on reactor-netty, which we do not support in native. I plan to improve the usage of the Azure SDK with Quarkus soon (~ September).

[cescoffier]

We would need specific extension, but you can pass this issue using:

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-identity</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>com.azure</groupId>
                    <artifactId>azure-core-http-netty</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-storage-blob</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>com.azure</groupId>
                    <artifactId>azure-core-http-netty</artifactId>
                </exclusion>
            </exclusions>
        </dependency>


<!--        <dependency>-->
<!--            <groupId>com.azure</groupId>-->
<!--            <artifactId>azure-core-http-vertx</artifactId>-->
<!--            <version>1.0.0-beta.1</version>-->
<!--        </dependency>-->

        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-support-azure-core-http-client-vertx</artifactId>
            <version>2.10.0</version>
        </dependency>

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-storage-queue</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>com.azure</groupId>
                    <artifactId>azure-core-http-netty</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-storage-file-share</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>com.azure</groupId>
                    <artifactId>azure-core-http-netty</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

Basically, exclude everything that use reactor netty.

[Cobas91]

That looks fine. The native build succeded. Now i test if everything is running! Thanks for your help

[cescoffier]

It may not be enough. I'm seeing lots of proxies, https needs to be registered...

I will start working on extensions after my very long GC pause (aka PTO)

[melloware]

Just a heads up Microsoft offers this now...

<dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-aot-graalvm-support-netty</artifactId>
            <version>1.0.0-beta.2</version>
        </dependency>
        <dependency>
            <groupId>net.java.dev.jna</groupId>
            <artifactId>jna-platform</artifactId>
            <version>5.12.1</version>
        </dependency>

However I did report this bug the problem is the version of Netty changed their signature of a class Microsoft used to override.

https://github.com/Azure/azure-sdk-for-java/issues/30332

I almost have it working for Quarkus but the issue above prevents it from working.

[tkb2022]

@melloware Hi! Quick question - are you still including the quarkus-maven-plugin inside of your project pom? Because I can get to the same stage as you are in the Azure #30332 issue after having added the azure-aot-graalvm-support, azure-aot-graalvm-support-netty, and jna-platform dependencies in my pom - it gives the error: Error: Could not find target method: static long com.azure.aot.graalvm.support.netty.implementation.features.TargetIoNettyUtilConcurrentScheduledFutureTask.initialNanoTime()

But only when I comment out the quarkus-maven-plugin. When that plugin is still in effect, I get the error like: Fatal error: java.lang.IllegalAccessError: superinterface check failed: class com.oracle.svm.core.genscavenge.BasicCollectionPolicies_OptionDescriptors (in unnamed module) cannot access class org.graalvm.compiler.options.OptionDescriptors (in module jdk.internal.vm.compiler) because module jdk.internal.vm.compiler does not export org.graalvm.compiler.options to unnamed module

Thanks for any insights you may have!

[melloware]

@tkb2022 you might want to look at this...Microsoft is working on native Quarkus support: https://github.com/quarkiverse/quarkus-azure-services

[cescoffier]

Closing this issue as https://github.com/quarkiverse/quarkus-azure-services is now handling the issue.

[albers]

We have implemented a more friendly HTTP client for the azure SDK (but I need to see how we can use it).

@cescoffier Is this client publicly available?

[cescoffier]

Yes, it should be part of the azure sdk now.

[albers]

@cescoffier I'm not sure. If I understand you correctly, we are talking about a new implementation of com.azure.core.http.HttpClientProvider.

Azure Core contains four implementations:

• com.azure.core.http.netty.NettyAsyncHttpClientProvider
• com.azure.core.http.vertx.VertxAsyncHttpClientProvider
• com.azure.core.http.okhttp.OkHttpAsyncClientProvider
• com.azure.core.http.jdk.httpclient.JdkHttpClientProvider

Do you mean VertxAsyncHttpClientProvider?

[cescoffier]

Yes, this one will be compatible with Quarkus.

[albers]

@cescoffier Thank you very much. I will try that one.