quarkus icon indicating copy to clipboard operation
quarkus copied to clipboard

Published 20 hours ago • verified publisher icon quarkusio

Quarkus initializing VertX cache even with vertx caching disabled

Open syr opened this issue 2 years ago • 5 comments

To comply to AWS ECS security rules we want to enforce read only filesystem access (https://docs.aws.amazon.com/config/latest/developerguide/ecs-containers-readonly-access.html)

This requires that vertx caching is disabled. However, no matter if defined by env var QUARKUS_VERTX_CACHING=false or app property: quarkus.vertx.caching=false

eventually, vertx caching is still active, causing the application to crash on startup

	Powered by Quarkus 2.10.3.Final

	2022-09-20T14:52:32.526+02:00	2022-09-20 12:52:32,526 WARN [io.qua.ver.cor.run.VertxCoreRecorder] (main) Unable to create Vert.x cache directory : /tmp/vertx-cache
	2022-09-20 12:52:32,526 WARN  [io.qua.ver.cor.run.VertxCoreRecorder] (main) Unable to make the Vert.x cache directory (/tmp/vertx-cache) world readable and writable
	2022-09-20 12:52:32,526 WARN [io.qua.ver.cor.run.VertxCoreRecorder] (main) Unable to make the Vert.x cache directory (/tmp/vertx-cache) world readable and writable
	2022-09-20T14:52:32.551+02:00	2022-09-20 12:52:32,549 WARN [io.qua.arc.impl] (main) An error occurred during delivery of the @BeforeDestroyed(ApplicationScoped.class) event: javax.enterprise.inject.CreationException: Synthetic bean instance for io.quarkus.scheduler.common.runtime.SchedulerContext not initialized yet: io_quarkus_scheduler_common_runtime_SchedulerContext_97d170e1550eee4afc0af065b78cda302a97674c
	2022-09-20T14:52:32.551+02:00	- a synthetic bean initialized during RUNTIME_INIT must not be accessed during STATIC_INIT
	2022-09-20T14:52:32.551+02:00	- RUNTIME_INIT build steps that require access to synthetic beans initialized during RUNTIME_INIT should consume the SyntheticBeansRuntimeInitBuildItem
	2022-09-20T14:52:32.551+02:00	at io.quarkus.scheduler.common.runtime.SchedulerContext_2480566c4c93757dc0e739a48013b17f9a0be445_Synthetic_Bean.create(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.scheduler.common.runtime.SchedulerContext_2480566c4c93757dc0e739a48013b17f9a0be445_Synthetic_Bean.get(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.scheduler.common.runtime.SchedulerContext_2480566c4c93757dc0e739a48013b17f9a0be445_Synthetic_Bean.get(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.CurrentInjectionPointProvider.get(CurrentInjectionPointProvider.java:60)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.quartz.runtime.QuartzScheduler_Bean.create(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.quartz.runtime.QuartzScheduler_Bean.create(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:111)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:35)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:32)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:26)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:32)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.quartz.runtime.QuartzScheduler_Observer_destroy_fb1039b288e54ae74a9e1acfa476b03b6c5afceb.notify(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:323)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:311)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:241)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.impl.ArcContainerImpl.shutdown(ArcContainerImpl.java:394)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.Arc.shutdown(Arc.java:57)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.arc.runtime.ArcRecorder$1.run(ArcRecorder.java:46)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.StartupContext.runAllInReverseOrder(StartupContext.java:84)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.StartupContext.close(StartupContext.java:73)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.Application.start(Application.java:101)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:103)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	2022-09-20T14:52:32.551+02:00	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	2022-09-20T14:52:32.551+02:00	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	2022-09-20T14:52:32.551+02:00	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	2022-09-20T14:52:32.551+02:00	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:60)
	2022-09-20T14:52:32.551+02:00	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:31)
	2022-09-20T14:52:32.605+02:00	2022-09-20 12:52:32,560 ERROR [io.qua.run.Application] (main) Failed to start application (with profile aws,dev): java.nio.file.FileSystemException: /tmp/vertx-cache: Read-only file system
	2022-09-20T14:52:32.605+02:00	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
	2022-09-20T14:52:32.605+02:00	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	2022-09-20T14:52:32.605+02:00	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	2022-09-20T14:52:32.605+02:00	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
	2022-09-20T14:52:32.605+02:00	at java.base/java.nio.file.Files.createDirectory(Files.java:700)
	2022-09-20T14:52:32.605+02:00	at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
	2022-09-20T14:52:32.605+02:00	at java.base/java.nio.file.Files.createDirectories(Files.java:793)
	2022-09-20T14:52:32.605+02:00	at io.vertx.core.file.impl.FileCache.setupCacheDir(FileCache.java:57)
	2022-09-20T14:52:32.605+02:00	at io.vertx.core.file.impl.FileCache.setupCache(FileCache.java:30)
	2022-09-20T14:52:32.605+02:00	at io.vertx.core.file.impl.FileResolverImpl.<init>(FileResolverImpl.java:66)
	2022-09-20T14:52:32.605+02:00	at io.vertx.core.impl.VertxBuilder.initFileResolver(VertxBuilder.java:334)
	2022-09-20T14:52:32.605+02:00	at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:274)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.vertx.core.runtime.VertxCoreRecorder.initialize(VertxCoreRecorder.java:237)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$VertxSupplier.get(VertxCoreRecorder.java:589)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$VertxSupplier.get(VertxCoreRecorder.java:571)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.vertx.http.runtime.VertxHttpRecorder.initializeRouter(VertxHttpRecorder.java:272)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.deployment.steps.VertxHttpProcessor$preinitializeRouter1141331088.deploy_0(Unknown Source)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.deployment.steps.VertxHttpProcessor$preinitializeRouter1141331088.deploy(Unknown Source)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runtime.Application.start(Application.java:101)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:103)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
	2022-09-20T14:52:32.605+02:00	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	2022-09-20T14:52:32.605+02:00	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	2022-09-20T14:52:32.605+02:00	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	2022-09-20T14:52:32.605+02:00	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	2022-09-20T14:52:32.605+02:00	at java.base/java.lang.reflect.Method.invoke(Method.java:568)

syr avatar Sep 20 '22 13:09 syr

/cc @gwenneg

quarkus-bot[bot] avatar Sep 20 '22 13:09 quarkus-bot[bot]

Any chance you can provide a sample project that reproduces this issue?

geoand avatar Sep 20 '22 13:09 geoand

I normally link a reproducer but in this case I don't know how to provide one as ECS is not included in AWS free tier.

Required to reproduce the exact problem: ECS service running one task where

  • task definition has parameter readonlyRootFilesystem=true
  • container image contains a basic/empty quarkus jvm application either with env var QUARKUS_VERTX_CACHING=false or with application property quarkus.vertx.caching=false

Will try if I can reproduce it locally, so I can maybe provide a reproducer.

syr avatar Sep 21 '22 16:09 syr

I think that if you have a reproducer, we can make things fail locally by tweaking the cache directory location

geoand avatar Sep 21 '22 16:09 geoand

Demonstrator (default generated project from code.quarkus.io and added quarkus.vertx.caching=false to app properties): https://github.com/syr/code-with-quarkus

Reproduce by docker build -f src/main/docker/Dockerfile.jvm -t quarkus/code-with-quarkus-jvm .

docker run -i --rm -p 8080:8080 --read-only quarkus/code-with-quarkus-jvm OR docker run -i --rm -p 8080:8080 --read-only -e QUARKUS_VERTX_CACHING=false quarkus/code-with-quarkus-jvm

stacktrace

docker run -i --rm -p 8080:8080 --read-only quarkus/code-with-quarkus-jvm
Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec  java -Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2022-09-21 17:13:30,253 WARN  [io.qua.ver.cor.run.VertxCoreRecorder] (main) Unable to create Vert.x cache directory : /tmp/vertx-cache
2022-09-21 17:13:30,284 WARN  [io.qua.ver.cor.run.VertxCoreRecorder] (main) Unable to make the Vert.x cache directory (/tmp/vertx-cache) world readable and writable
2022-09-21 17:13:30,304 ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): java.nio.file.FileSystemException: /tmp/vertx-cache: Read-only file system
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
        at java.base/java.nio.file.Files.createDirectory(Files.java:700)
        at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
        at java.base/java.nio.file.Files.createDirectories(Files.java:793)
        at io.vertx.core.file.impl.FileCache.setupCacheDir(FileCache.java:57)
        at io.vertx.core.file.impl.FileCache.setupCache(FileCache.java:30)
        at io.vertx.core.file.impl.FileResolverImpl.<init>(FileResolverImpl.java:66)
        at io.vertx.core.impl.VertxBuilder.initFileResolver(VertxBuilder.java:334)
        at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:274)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder.initialize(VertxCoreRecorder.java:239)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder$VertxSupplier.get(VertxCoreRecorder.java:591)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder$VertxSupplier.get(VertxCoreRecorder.java:573)
        at io.quarkus.vertx.http.runtime.VertxHttpRecorder.initializeRouter(VertxHttpRecorder.java:287)
        at io.quarkus.deployment.steps.VertxHttpProcessor$preinitializeRouter1141331088.deploy_0(Unknown Source)
        at io.quarkus.deployment.steps.VertxHttpProcessor$preinitializeRouter1141331088.deploy(Unknown Source)
        at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
        at io.quarkus.runtime.Application.start(Application.java:101)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:110)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:69)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:122)
        at io.quarkus.runner.GeneratedMain.main(Unknown Source)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:60)
        at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:31)

syr avatar Sep 21 '22 17:09 syr

To overcome your problem you essentially also need to set QUARKUS_VERTX_CLASSPATH_RESOLVING=false.

https://github.com/quarkusio/quarkus/pull/28142 will take care of the warning messages you see in the output.

geoand avatar Sep 22 '22 06:09 geoand

Same issue now with latest Quarkus 2.13.3.Final, when using quarkus-smallrye-graphql-client even with

quarkus.vertx.caching=false
quarkus.vertx.classpath-resolving=false

I get error

 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2022-10-27 09:35:26,871 ERROR [io.qua.run.Application] (main) Failed to start application (with profile prod): java.nio.file.FileSystemException: /tmp/vertx-cache-81092607-e0a3-49c4-8734-213cba4c7b4d: Read-only file system
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
        at java.base/java.nio.file.Files.createDirectory(Files.java:700)
        at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
        at java.base/java.nio.file.Files.createDirectories(Files.java:793)
        at io.vertx.core.file.impl.FileCache.setupCacheDir(FileCache.java:58)
        at io.vertx.core.file.impl.FileCache.setupCache(FileCache.java:31)
        at io.vertx.core.file.impl.FileResolverImpl.<init>(FileResolverImpl.java:67)
        at io.vertx.core.impl.VertxBuilder.initFileResolver(VertxBuilder.java:334)
        at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:274)
        at io.vertx.core.Vertx.vertx(Vertx.java:87)
        at io.vertx.core.Vertx.vertx(Vertx.java:77)
        at io.smallrye.graphql.client.vertx.dynamic.VertxDynamicGraphQLClientBuilder.build(VertxDynamicGraphQLClientBuilder.java:133)
        at io.smallrye.graphql.client.impl.dynamic.cdi.NamedDynamicClients.lambda$getClient$0(NamedDynamicClients.java:41)
        at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1220)
        at io.smallrye.graphql.client.impl.dynamic.cdi.NamedDynamicClients.getClient(NamedDynamicClients.java:40)
        at io.smallrye.graphql.client.impl.dynamic.cdi.NamedDynamicClients_ProducerMethod_getClient_1a472307edb2c0c36e1f2fc6b46eeac419bdaed8_Bean.create(Unknown Source)
        at io.smallrye.graphql.client.impl.dynamic.cdi.NamedDynamicClients_ProducerMethod_getClient_1a472307edb2c0c36e1f2fc6b46eeac419bdaed8_Bean.get(Unknown Source)
        at io.smallrye.graphql.client.impl.dynamic.cdi.NamedDynamicClients_ProducerMethod_getClient_1a472307edb2c0c36e1f2fc6b46eeac419bdaed8_Bean.get(Unknown Source)
        at io.quarkus.arc.impl.CurrentInjectionPointProvider.get(CurrentInjectionPointProvider.java:62)
        at client.StarWarsResource_Bean.create(Unknown Source)
        at client.StarWarsResource_Bean.create(Unknown Source)

Maybe graphql client does not work without vertx cache or it is missing to respect

quarkus.vertx.caching=false
quarkus.vertx.classpath-resolving=false

Reproucer based on microprofile-graphql-client-quickstart: https://github.com/syr/code-with-quarkus

Not sure if this issue should be reopened or if I should create another one.

syr avatar Oct 27 '22 09:10 syr

Not sure if this issue should be reopened or if I should create another one.

Please create a new one

geoand avatar Oct 27 '22 09:10 geoand