azure-sdk-for-java
azure-sdk-for-java copied to clipboard
[BUG] Occurs SecurityException when building native image based on Spring Cloud Azure Native support
Describe the bug
When using native tools to build, the failure Fatal error: java.lang.SecurityException: class "com.azure.spring.cloud.autoconfigure.compatibility.AzureCompatibilityVerifierAutoConfiguration"'s signer information does not match signer information of other classes in the same package
occurs.
Exception or Stack Trace
Fatal error: java.lang.SecurityException: class "com.azure.spring.cloud.autoconfigure.compatibility.AzureCompatibilityVerifierAutoConfiguration"'s signer information does not match signer information of other classes in the same package
at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1151)
at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:906)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1015)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at com.oracle.svm.hosted.ImageClassLoader.forName(ImageClassLoader.java:285)
at com.oracle.svm.hosted.ImageClassLoader.findClass(ImageClassLoader.java:278)
at com.oracle.svm.hosted.config.ReflectionRegistryAdapter.resolveType(ReflectionRegistryAdapter.java:68)
at com.oracle.svm.core.configure.ReflectionConfigurationParser.parseClass(ReflectionConfigurationParser.java:94)
at com.oracle.svm.core.configure.ReflectionConfigurationParser.parseClassArray(ReflectionConfigurationParser.java:77)
at com.oracle.svm.core.configure.ReflectionConfigurationParser.parseAndRegister(ReflectionConfigurationParser.java:72)
at com.oracle.svm.hosted.config.ConfigurationParserUtils.doParseAndRegister(ConfigurationParserUtils.java:127)
at com.oracle.svm.hosted.config.ConfigurationParserUtils.lambda$parseAndRegisterConfigurations$3(ConfigurationParserUtils.java:113)
at java.base/java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:212)
at com.oracle.svm.hosted.config.ConfigurationParserUtils$1.tryAdvance(ConfigurationParserUtils.java:106)
at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:326)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:491)
at java.base/java.util.stream.IntPipeline.sum(IntPipeline.java:449)
at com.oracle.svm.hosted.config.ConfigurationParserUtils.parseAndRegisterConfigurations(ConfigurationParserUtils.java:115)
at com.oracle.svm.reflect.hosted.ReflectionFeature.duringSetup(ReflectionFeature.java:185)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$setupNativeImage$16(NativeImageGenerator.java:847)
at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:74)
at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:847)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:534)
at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:494)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:426)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:587)
at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:126)
at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:617)
To Reproduce Run sample issue-azure-storage-native
Code Snippet Ref the sample project.
Expected behavior Build and exec successfully.
Setup (please complete the following information):
- OS: [Windows 11]
- IDE: [IntelliJ]
- Library/Libraries: [com.azure.spring:spring-cloud-azure-starter-storage-blob:4.1.0, com.azure.spring:spring-cloud-azure-native-configuration:4.0.0-beta.1]
- Java version: [11]
- Frameworks: [Spring Boot 2.6.6]
Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
- [x] Bug Description Added
- [x] Repro Steps Added
- [x] Setup information Added
I opened a new issue https://github.com/spring-projects-experimental/spring-native/issues/1699 on the Spring Native side for help.
FYI the related Spring Framework 6 issue is spring-projects/spring-framework#29019.
Do you have a workaround for this issue or is it 100% blocking? Does it happen only with native build tools or also with Buildpacks?
It is currently blocked, and both native build tool and Buildpacks will encounter it.
Could you please share more on which JARs are signed and when? We would be interested to know if only Azure library jars are signed when they are built (so the fact they are signed is something we can detected during the build) or are all JARs signed during the deployment on Azure?
All the jars from Azure side will be signed before they are published to Maven Central, this action is built in the release pipeline for each Azure Java library, the libraries with group id com.azure.spring
and artifact id prefix spring-cloud-azure-xxx
are maintained and released by our team, like: spring-cloud-azure-autoconfigure
, spring-cloud-azure-starter
, see more from https://mvnrepository.com/artifact/com.azure.spring.
all JARs signed during the deployment on Azure
I understand the deployment on Azure is that deploy the Spring Boot application jar to Azure service, like Azure App Services, etc, this jar does not need to be signed.
Juergen Hoeller has provided the following feedback about this issue here:
After a lot of consideration, we have realized that this is a problem that is not practical to solve at the core framework level. Our generated configuration needs to have access to package-local elements in common scenarios, not least of it all in order to avoid unnecessary reflection. For that reason, we decided to preserve our package-local generation approach.
If a separate jar with a split package arrangement or different jar signatures turns out to be an issue (also e.g. in the module system), the application build may combine them into a single jar that contains both the original classes and the generated configuration. Alternatively, the application build may also simply remove the jar signature before proceeding.
So I would like to discuss with you to try to find a workaround that works for Azure use case.
The real issue here is not the validation of JAR integrity with the signature which can happen separately via jarsigner -verify foo.jar
, but the fact that when JAR are signed, the JVM don't accept additional classes in the same package.
So what are the possible workarounds:
- Disabling JAR signature verification as described here
- Combine all JARs into a single one
- Explode the dependency JARs before using it (signature are only verified for JARs)
- Leverage Gradle artifact transformation to remove the signature during the build
Do you think we could find a workaround suitable for Azure?
Thanks @sdeleuze. Did you mean disabling the JAR signature verification during the native build? Could you elaborate on combine all JARs into a single one
? When and what should we combine? What does the explode
mean in option 3?
Did you mean disabling the JAR signature verification during the native build? Yes.
On JVM side, running from that fat JAR works since no verification is done on the embedded JARs, only running in exploded mode when AOT is enabled is broken, but using that path is pretty involved (AOT processing needs to be enable + -Dspring.aot.enabled=true
net to be set) and that's not really a first class use case, so I think that's ok at least for now to not support this.
Could you elaborate on combine all JARs into a single one? I mean using build plugins to create one JAR by merging all the classes and resources of the application and its dependencies. I think https://imperceptiblethoughts.com/shadow/ does something like that.
What does the explode mean in option 3? That means extracting each dependency in a directory and use those directories instead of the dependencies JAR as the classpath.
The issue with option 2 and 3 is that I am not sure that will integrate properly with Native Build Tools and Buildpacks.
Option 1 is the simpler but require creating a local file where the native-image
compiler runs (local machine for Native Build Tools and container for Buildpacks).
Option 4 is IMO the the more robust since it produces artifact that will work for all use case, should be easy with Gradle but not sure how to handle it with Maven.
Maybe we can ask GraalVM team to add native-image
option to disable JAR signature verification with a simple option and for the time being we move forward by creating the file and configuring it manually.
Please let me know what would be the best path for you.
@sdeleuze thanks for your update!
I tried option 1 using the Spring Cloud Azure 4.10, it resulted in the below same exception.
-
If not configured the
custom.security
file, the build failed in the first phrase[1/7] Initializing
.Not configured the costom.security file
</p>
-
If configured the
custom.security
file which has already added a linejdk.jar.disabledAlgorithms=MD2, MD5, RSA, DSA
, the build failed after the phrase[2/7] Performing analysis... [*********]
. From the exception stack, seems that the later command with java (should be) does not take thecustome.security
file configuration, so it still executed the signature verification.Configured the costom.security file
</p>
I checked with the reproducer sample https://github.com/mhalbritter/spring-aot-jarsigner-reproducer, after the custome.security
file was configured, the exception is the same as before.
Console log from command `gradlew.bat bootRun -Djava.security.properties=custom-java17.security`
@sdeleuze Could you help to give more suggestions for option 1? I think this workaround is simpler for users.
New update: The sample spring-aot-jarsigner-reproducer works through the code setting systemProperty('java.security.properties', 'D:\\xxx\\java17.security')
But when using native compile, the exception seems the same with my local testing.
[2/7] Performing analysis... [*] (24.3s @ 1.33GB)
5,951 (84.08%) of 7,078 classes reachable
8,148 (59.99%) of 13,582 fields reachable
27,107 (52.88%) of 51,266 methods reachable
248 classes, 118 fields, and 873 methods registered for reflection
3 native libraries: crypt32, ncrypt, psapi
Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing com.example.signerdemo.SignerDemoApplication__ApplicationContextInitializer.initialize(org.springframework.context.support.GenericApplicationContext)
Parsing context:
at com.example.signerdemo.SignerDemoApplication__ApplicationContextInitializer.initialize(SignerDemoApplication__ApplicationContextInitializer.java:16)
at com.example.signerdemo.SignerDemoApplication__ApplicationContextInitializer.initialize(SignerDemoApplication__ApplicationContextInitializer.java:13)
at org.springframework.boot.context.config.DelegatingApplicationContextInitializer.applyInitializers(DelegatingApplicationContextInitializer.java:107)
at org.springframework.boot.context.config.DelegatingApplicationContextInitializer.applyInitializerClasses(DelegatingApplicationContextInitializer.java:88)
at org.springframework.boot.context.config.DelegatingApplicationContextInitializer.initialize(DelegatingApplicationContextInitializer.java:56)
at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:72)
at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:152)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:109)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:558)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:635)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Error loading a referenced type: java.lang.SecurityException: class "dependency.DependencyAutoConfiguration__BeanDefinitions"'s signer information does not match signer information of other classes in the same package
at parsing com.example.signerdemo.SignerDemoApplication__BeanFactoryRegistrations.registerBeanDefinitions(SignerDemoApplication__BeanFactoryRegistrations.java:46)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2506)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:105)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3367)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3319)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3164)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1138)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:79)
at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:261)
at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:135)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:685)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:180)
at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1162)
at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1145)
at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:1003)
at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:957)
at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:817)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:176)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:343)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
... 13 more
Caused by: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Error loading a referenced type: java.lang.SecurityException: class "dependency.DependencyAutoConfiguration__BeanDefinitions"'s signer information does not match signer information of other classes in the same package
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:153)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:159)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SubstrateClassInitializationPlugin.loadReferencedType(SubstrateClassInitializationPlugin.java:58)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.loadReferenceType(BytecodeParser.java:4250)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.maybeEagerlyResolve(BytecodeParser.java:4232)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.maybeEagerlyResolve(SharedGraphBuilderPhase.java:153)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.lookupMethod(BytecodeParser.java:4172)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1636)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5224)
at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3359)
... 38 more
Caused by: java.lang.SecurityException: class "dependency.DependencyAutoConfiguration__BeanDefinitions"'s signer information does not match signer information of other classes in the same package
at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1158)
at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:902)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1010)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.resolveTypeInPool(Native Method)
------------------------------------------------------------------------------------------------------------------------
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotConstantPool.loadReferencedType(HotSpotConstantPool.java:865)
at java.base/jdk.internal.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:142)
... 47 more
1.5s (3.9% of total time) in 16 GCs | Peak RSS: 1.95GB | CPU load: 2.73
For gradle bootRun
you should configure something like that in app/build.gradle
:
bootRun {
systemProperty("java.security.properties", "/home/seb/playground/spring-aot-jarsigner-reproducer/custom.security")
systemProperty('spring.aot.enabled', 'true')
}
See related Spring Boot documentation. It should be even possible to create the custom.security
file in build.gradle
.
For native image, you can configure the application module in app/build.gradle
with
graalvmNative {
binaries {
main {
buildArgs('-Djava.security.properties=/home/seb/playground/spring-aot-jarsigner-reproducer/custom.security')
}
}
}
With custom.security
content behing:
jdk.jar.disabledAlgorithms=MD2, MD5, RSA, DSA
Works for me on spring-aot-jarsigner-reproducer
The only thing I am not sure yet is how to make this supported with Buildpacks, but let's for now see if it works for you with Native Build Tools.
Thanks @sdeleuze, it works for now with the Native Build Tools.
Below are my testing steps in Windows 11:
-
Locate your file
java.security
, like D:\xxx\graalvm-ce-java11-22.0.0.2\conf\security\java.security. -
Comment out the default configuration, it maybe is in line 697
-
Add a new value
MD2, MD5, RSA, DSA
for the commented keyjdk.jar.disabledAlgorithms
-
Then go ahead with the native executable building.
I will test the other possible workarounds and update you here if available.
Good to know, I have updated my comment to show the content of the custom.security
file which should provide the same result than the direct JDK modification specified above.
We are open to discuss Buildpacks support for this.