quarkus
quarkus copied to clipboard
quarkusio
Build fails with static OpenAPI file in META-INF folder
Describe the bug
When we put a static OpenAPI file in META-INF folder (the default location), the build fails with this error:
C:\projects\acctmgt-back\target\acctmgmt-back.jar -> C:\projects\acctmgt-back\target\acctmgmt-back.jar.original: The process cannot access the file because it is being used by another process.
If we remove the static OpenAPI file then the build is successful.
So apparently Quarkus has a file lock on this file.
We are using 999-SNAPSHOT to pickup some other recent changes.
Note we are building on Windows and generating an uber-jar type.
Expected behavior
Build should not fail
Actual behavior
Build fails
How to Reproduce?
No response
Output of uname -a or ver
No response
Output of java -version
java 21.0.4 2024-07-16 LTS
Quarkus version or git rev
No response
Build tool (ie. output of mvnw --version or gradlew --version)
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Additional information
No response
/cc @EricWittmann (openapi), @MikeEdgar (openapi), @phillip-kruger (openapi)
![quarkus-bot[bot] avatar](https://avatars.githubusercontent.com/in/90234?v=4 "Published by a pub.dev verified publisher"){kind=small}
Okay I have created a reproducer and attached it here. The build will fail with the reported error unless you remove the static openapi.yaml file.
Also we would like to be able to load the static openapi.yaml from a different directory not in the classpath even, but that does not work for us. Can you explain the limits of the additional-docs-directory property? E.g. where does that property assume the root path is?
@dhoffer are you having the problem with a simple mvn package? I tried that with your reproducer (on Linux) without any issues. Do you get any stack trace or something besides the message about the file being accessed by another process? Also, just confirming you don't have the file open in an IDE or text editor, etc.
I'm building from CMD, no IDE open to project. Here is the results of mvn -X clean package
I am sort of assuming this is a Windows issue.
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project code-with-quarkus: Failed to build quarkus application: java.nio.file.FileSystemException: C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar -> C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar.original: The process cannot access the file because it is being used by another process -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project code-with-quarkus: Failed to build quarkus application
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
at java.lang.reflect.Method.invoke (Method.java:580)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.maven.plugin.MojoExecutionException: Failed to build quarkus application
at io.quarkus.maven.BuildMojo.doExecute (BuildMojo.java:187)
at io.quarkus.maven.QuarkusBootstrapMojo.execute (QuarkusBootstrapMojo.java:171)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
at java.lang.reflect.Method.invoke (Method.java:580)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: java.io.UncheckedIOException: java.nio.file.FileSystemException: C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar -> C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar.original: The process cannot access the file because it is being used by another process
at io.quarkus.maven.BuildMojo.doExecute (BuildMojo.java:161)
at io.quarkus.maven.QuarkusBootstrapMojo.execute (QuarkusBootstrapMojo.java:171)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
at java.lang.reflect.Method.invoke (Method.java:580)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: java.nio.file.FileSystemException: C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar -> C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar.original: The process cannot access the file because it is being used by another process
at sun.nio.fs.WindowsException.translateToIOException (WindowsException.java:92)
at sun.nio.fs.WindowsException.rethrowAsIOException (WindowsException.java:103)
at sun.nio.fs.WindowsFileCopy.move (WindowsFileCopy.java:414)
at sun.nio.fs.WindowsFileSystemProvider.move (WindowsFileSystemProvider.java:291)
at java.nio.file.Files.move (Files.java:1430)
at io.quarkus.maven.BuildMojo.doExecute (BuildMojo.java:159)
at io.quarkus.maven.QuarkusBootstrapMojo.execute (QuarkusBootstrapMojo.java:171)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
at java.lang.reflect.Method.invoke (Method.java:580)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
What is the status of this? This is a blocking issue for us regarding upgrading to Quarkus 3 and Microprofile OpenAPI. All developers build test with Windows only our CI system is Linux so we have to have good Windows support.
I don't have a Windows machine to try this one. Does this same thing happen with the latest Quarkus release, rather than the SNAPSHOT?
I'm shooting in the dark as I don't have a Windows machine either but from what I can see the OpenAPI extension code is not very careful about closing the opened InputStreams (unless I'm mistaken, in which case sorry for the noise).
A good example of this issue is the following code:
https://github.com/quarkusio/quarkus/blob/539705ff061169f3b9b4f9d05fad769304e21446/extensions/smallrye-openapi/deployment/src/main/java/io/quarkus/smallrye/openapi/deployment/SmallRyeOpenApiProcessor.java#L1011-L1026
It's cute but from what I can see it's opening InputStreams and they are never closed. Now I could be mistaken and the streams might actually get closed somewhere in SmallRye OpenAPI but I looked around and couldn't find where it's done at a quick glance.
If I'm actually correct, I think it would be worth auditing this area a bit. And by area, I mean anything opening an InputStream in this code, there might be other places.
Windows is a lot more sensitive to file locking but in this case it might actually report an actual issue in our code.
@MikeEdgar could you make sure I'm not hallucinating?
I would have expected them to be closed via the InputStreamReader used within the builder [1] but I must be overlooking something. I'll keep digging.
[1] https://github.com/smallrye/smallrye-open-api/blob/1a7ece62bc367422729fa6d9cd8300fab8de857a/core/src/main/java/io/smallrye/openapi/api/SmallRyeOpenAPI.java#L566-L573
Yeah, that's why I asked you. It sure seems like these particular ones should be fine.
It might be a false alarm but I think something somewhere keeps a lock. I'll have another look if I can find something.
I had another look at the report and second question: are you closing the OpenApiStaticFile that are opened? Because it seems to be triggered by the addition of a META-INF/openapi.yaml and I can see you have an InputStream in OpenApiStaticFile .
Yes, that is a few lines up [1] from the additional/custom static file.
[1] https://github.com/smallrye/smallrye-open-api/blob/3.13.0/core/src/main/java/io/smallrye/openapi/api/SmallRyeOpenAPI.java#L546-L560
It couldn't possibly be from the META-INF/openapi.yaml being a HotDeploymentWatchedFileBuildItem could it?
This shouldn't affect things outside of dev mode but you never know. Let me try another strategy.
@dhoffer , in case it was lost can you confirm if this same thing happens with the latest Quarkus release, specifically 3.15.1, rather than the SNAPSHOT? There have been changes in OpenAPI since then and that would help rule out or confirm if this is a regression.
I had a look at a heap dump just before the uberjar was built and couldn't find anything obvious e.g. no suspicious InputStream being open in the heap.
So really not a clue what's going on.
Just to mention it, seems odd that the exception mentions that jars, as if one of them is open in a file explorer or something (?). I would think a lock on the openapi.yaml file would be reported as such. This makes it seem as if the problem is entirely with the jar or the jar.original.
java.nio.file.FileSystemException: C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar -> C:\git\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar.original: The process cannot access the file because it is being used by another process
@gastaldi I think you have a Windows box around? Could you check if you can reproduce it with the reproducer? Be careful, it refers to io.quarkus.platform but a snapshot, probably a good idea to change that to io.quarkus.
Yeah so it could also be a file actually loaded from the jar. But I don't think we load anything from the produced jar so this looks extremely odd.
Let's see if it can be reproduced independently.
@dhoffer , one more thing - you've mentioned elsewhere that you're using filters, etc. Can you confirm that none of your OASFilter implementations are loading META-INF/openapi.json and not closing it?
I tried the reproducer in my Windows machine and I see the error described in https://github.com/quarkusio/quarkus/issues/43940#issuecomment-2422932469
@gastaldi @gsmet @MikeEdgar Yes we do use OASFilter but we don't touch the META-INF/openapi.json file, but just to be certain I have removed OASFilter usage in my reproducer. This is what I see building with various Quarkus versions.
999-SNAPSHOT Fails 3.16.0.CR1 Fails 3.15.1 Works
I reproduced it in github using the reproducer project, so I'll try to dig in that way.
There seems to be something very odd occurring with an InputStream obtained by the classloader. I've disabled the entire Smallrye OpenAPI builder method that handles static files [1] and added a simple lookup of an existing static file, opening the stream in a try-with-resources, and the locked file still occurs. Note, that removing the block of code below from the linked method [1] results in a passing/green build. I.e., any static file passed to the OpenAPI builder is ignored.
This must be something lower-level than the OpenAPI processor. What do you think @gsmet @gastaldi ?
I have the build running on a GH Windows image in a repo with the reproducer project [2]. The CI there builds a branch of smallrye-open-api in my fork and overrides the version managed by Quarkus to be the local snapshot.
URL metaInfYaml = Thread.currentThread().getContextClassLoader().getResource("META-INF/openapi.yaml");
if (metaInfYaml != null) {
try (InputStream stream = metaInfYaml.openStream()) {
} catch (IOException e) {
String msg = "IOException reading static file: " + metaInfYaml;
throw new OpenApiRuntimeException(msg, e);
}
}
[1] https://github.com/MikeEdgar/smallrye-open-api/blob/8557535dc2a39b88cb694d80c2e6da71ad55bbb9/core/src/main/java/io/smallrye/openapi/api/SmallRyeOpenAPI.java#L560-L636 [2] https://github.com/MikeEdgar/openapi-windows-npe/actions
I'd guess it's another problem in the bowels of the QuarkusClassLoader or its PathTree implementation. Maybe @aloubyansky has an idea of where to start looking? Otherwise I can try to figure it out tomorrow.
Perhaps try with -DskipOriginalJarRename on the command line or add it to the quarkus-maven-plugin configuration in the POM to avoid moving the file.
I don't have a Windows machine to test but here is what I would try first https://github.com/quarkusio/quarkus/compare/main...aloubyansky:quarkus:build-file-move
That will close the application (along with its classloaders) before moving the file.
@dhoffer would you be able to give it a try?
@aloubyansky I gave your branch a try and I still get the same error:
[INFO] --- jar:3.4.1:jar (default-jar) @ code-with-quarkus ---
[INFO] Building jar: C:\Users\ggastald\Downloads\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- quarkus:999-SNAPSHOT:build (default) @ code-with-quarkus ---
[INFO] [io.quarkus.smallrye.openapi] OpenAPI json saved: C:\Users\ggastald\Downloads\code-with-quarkus\target\generated-openapi\openapi.json
[INFO] [io.quarkus.smallrye.openapi] OpenAPI yaml saved: C:\Users\ggastald\Downloads\code-with-quarkus\target\generated-openapi\openapi.yaml
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building uber jar: C:\Users\ggastald\Downloads\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT-runner.jar
[WARNING] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Dependencies with duplicate files detected. The dependencies [org.jctools:jctools-core::jar:4.0.5[paths: C:\Users\ggastald\.m2\repository\org\jctools\jctools-core\4.0.5\jctools-core-4.0.5.jar;], io.netty:netty-common::jar:4.1.111.Final[paths: C:\Users\ggastald\.m2\repository\io\netty\netty-common\4.1.111.Final\netty-common-4.1.111.Final.jar;]] contain duplicate files, e.g. META-INF/maven/org.jctools/jctools-core/pom.xml
[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 13632ms
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33.217 s
[INFO] Finished at: 2024-10-29T22:59:33-03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project code-with-quarkus: Failed to build quarkus application: java.nio.file.FileSystemException: C:\Users\ggastald\Downloads\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar -> C:\Users\ggastald\Downloads\code-with-quarkus\target\code-with-quarkus-1.0.0-SNAPSHOT.jar.original: The process cannot access the file because it is being used by another process -> [Help 1]