quarkus icon indicating copy to clipboard operation
quarkus copied to clipboard

Add GeneratedStaticResourceProcessor

Open mcruzdev opened this issue 10 months ago • 9 comments

Fixes #39735

cc @FroMage @cescoffier @ppalaga

mcruzdev avatar Apr 04 '24 21:04 mcruzdev

In https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/deployment/builditem/GeneratedResourceBuildItem.java#L9-L11

We should change the comment to recommend using GeneratedStaticResourceBuildItem which takes care of this.

ia3andy avatar Apr 05 '24 08:04 ia3andy

In AdditionalStaticResourceBuildItem we should also explain that it's not made for generated resources and to use GeneratedStaticResourceBuildItem instead.

AdditionalStaticResourceBuildItem should only be for prod, and maybe we should even remove the processing for dev (to avoid any possible conflict): https://github.com/quarkusio/quarkus/blob/main/extensions/vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/StaticResourcesProcessor.java#L49-L51

ia3andy avatar Apr 05 '24 08:04 ia3andy

In any case, this is going to need tests.

FroMage avatar Apr 05 '24 08:04 FroMage

🙈 The PR is closed and the preview is expired.

github-actions[bot] avatar Apr 12 '24 00:04 github-actions[bot]

@mcruzdev maybe I am missing something but I don't understand why you are creating the generated files on disk? As soon as a https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/deployment/builditem/GeneratedResourceBuildItem.java is created, it means you will have the files available in the classpath at runtime, so in dev, you just have to in a buildstep to create a Set of the available generated resources that you pass to the recorder and then in the ClassPathStaticHandler, you use that map to know if the path should be handled or not (next), and if it should the you can get the content byte[] directly from the classpath.

Again maybe I am missing something?

ia3andy avatar Apr 12 '24 09:04 ia3andy


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 87416b3d5333ec41fde6d88b58f99f7a191b7569.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Initial JDK 17 Build Build Failures Logs Raw logs :mag:

You can consult the Develocity build scans.

Failures

:gear: Initial JDK 17 Build #

- Failing: extensions/vertx-http/deployment-spi 
! Skipped: devtools/bom-descriptor-json devtools/cli docs and 418 more

:package: extensions/vertx-http/deployment-spi

Failed to execute goal net.revelc.code.formatter:formatter-maven-plugin:2.23.0:validate (default) on project quarkus-vertx-http-deployment-spi: File '/home/runner/work/quarkus/quarkus/extensions/vertx-http/deployment-spi/src/main/java/io/quarkus/vertx/http/deployment/spi/GeneratedStaticResourceBuildItem.java' has not been previously formatted. Please format file (for example by invoking `mvn -f extensions/vertx-http/deployment-spi net.revelc.code.formatter:formatter-maven-plugin:2.23.0:format`) and commit before running validation!

quarkus-bot[bot] avatar Apr 17 '24 07:04 quarkus-bot[bot]

Could we add a DevModeTest similar to this: https://github.com/quarkiverse/quarkus-web-bundler/blob/main/deployment/src/test/java/io/quarkiverse/web/bundler/test/WebBundlerDevModeTest.java

ia3andy avatar Apr 17 '24 07:04 ia3andy

This is getting really close :)

ia3andy avatar Apr 19 '24 14:04 ia3andy

Nice contribution!

I had a small look as I was wondering what it was doing and I added a few comments. Probably worth waiting for @cescoffier 's feedback before acting on the ones that are not obvious.

I changed the documentation being more direct to avoid confusion, let me know WDYT?

mcruzdev avatar Apr 21 '24 02:04 mcruzdev

@mcruzdev you also need to edit this: https://github.com/quarkusio/quarkus/blob/2bfd9cbb63b4511a36edc981ff8886f6b32a05d5/extensions/vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/devmode/NotFoundProcessor.java#L58

In order to display the generated routes in the 404

ia3andy avatar May 07 '24 09:05 ia3andy


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit cdb920b0db5fb0eaaa4ebfcb46ae86170649c38b.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Initial JDK 17 Build Build Failures Logs Raw logs :mag:

You can consult the Develocity build scans.

Failures

:gear: Initial JDK 17 Build #

- Failing: extensions/vertx-http/runtime 
! Skipped: devtools/bom-descriptor-json devtools/cli docs and 538 more

:package: extensions/vertx-http/runtime

Failed to execute goal net.revelc.code.formatter:formatter-maven-plugin:2.23.0:validate (default) on project quarkus-vertx-http: File '/home/runner/work/quarkus/quarkus/extensions/vertx-http/runtime/src/main/java/io/quarkus/vertx/http/runtime/handlers/DevClasspathStaticHandler.java' has not been previously formatted. Please format file (for example by invoking `mvn -f extensions/vertx-http/runtime net.revelc.code.formatter:formatter-maven-plugin:2.23.0:format`) and commit before running validation!

quarkus-bot[bot] avatar May 09 '24 21:05 quarkus-bot[bot]

This is an awesome contribution by @mcruzdev on his free time 🙏🚀

This PR looks good to me, I'll test it with the web-bundler today. @gsmet @cescoffier @FroMage please give your final review on the code part. It would be great if we could merge and increment so if there are some issue which can be fixed in other following PRs, let's merge this one and create issues.

ia3andy avatar May 10 '24 07:05 ia3andy


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit cfa112d47f40d315495bd077078f1d7476c07fc5.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Native Tests - Security3 Build Failures Logs Raw logs :mag:

Full information is available in the Build summary check run. You can consult the Develocity build scans.

Failures

:gear: Native Tests - Security3 #

- Failing: integration-tests/keycloak-authorization 

:package: integration-tests/keycloak-authorization

io.quarkus.it.keycloak.NativeAdminClientITCase.testGetNewRealm - History - More details - Source on GitHub

java.lang.RuntimeException: java.util.concurrent.CompletionException: java.lang.RuntimeException: Unable to start Quarkus test resource class io.quarkus.it.keycloak.KeycloakLifecycleManager
	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.throwBootFailureException(QuarkusIntegrationTestExtension.java:372)
	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.beforeEach(QuarkusIntegrationTestExtension.java:117)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeException: Unable to start Quarkus test resource class io.quarkus.it.keycloak.KeycloakLifecycleManager
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)

Flaky tests - Develocity

:gear: JVM Tests - JDK 17

:package: extensions/smallrye-reactive-messaging-kafka/deployment

io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

  • Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

:gear: Maven Tests - JDK 17 Windows

:package: integration-tests/maven

io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch - History

  • Cannot delete directory: target\test-classes\projects\project-generation - java.lang.RuntimeException
java.lang.RuntimeException: Cannot delete directory: target\test-classes\projects\project-generation
	at io.quarkus.maven.it.MojoTestBase.initEmptyProject(MojoTestBase.java:72)
	at io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch(CreateProjectMojoIT.java:60)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: org.apache.commons.io.IOExceptionList: 1 exception(s): [org.apache.commons.io.IOIndexedException: IOException #0: Cannot delete file: target\test-classes\projects\project-generation\build-create-project-generation.log]
	at org.apache.commons.io.IOExceptionList.checkEmpty(IOExceptionList.java:50)

io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch - History

  • Cannot delete directory: target\test-classes\projects\project-generation - java.lang.RuntimeException
java.lang.RuntimeException: Cannot delete directory: target\test-classes\projects\project-generation
	at io.quarkus.maven.it.MojoTestBase.initEmptyProject(MojoTestBase.java:72)
	at io.quarkus.maven.it.CreateProjectMojoIT.testProjectGenerationFromScratch(CreateProjectMojoIT.java:60)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: org.apache.commons.io.IOExceptionList: 1 exception(s): [org.apache.commons.io.IOIndexedException: IOException #0: Cannot delete file: target\test-classes\projects\project-generation\build-create-project-generation.log]
	at org.apache.commons.io.IOExceptionList.checkEmpty(IOExceptionList.java:50)

quarkus-bot[bot] avatar May 10 '24 11:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 6ed8c3bc1c3ccc7a317f20cec49485790d4c60a9.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
:heavy_check_mark: JVM Tests - JDK 17 Logs Raw logs :mag:
:heavy_check_mark: JVM Tests - JDK 21 Logs Raw logs :mag:
JVM Tests - JDK 17 Windows Build Failures Logs Raw logs :mag:

You can consult the Develocity build scans.

Failures

:gear: JVM Tests - JDK 17 Windows #

- Failing: integration-tests/rest-client integration-tests/rest-client-reactive 

:package: integration-tests/rest-client

Failed to execute goal uk.co.automatictester:truststore-maven-plugin:3.0.0:generate-truststore (self-signed-truststore) on project quarkus-integration-test-rest-client: Execution self-signed-truststore of goal uk.co.automatictester:truststore-maven-plugin:3.0.0:generate-truststore failed: Unable to establish TLS connection with: self-signed.badssl.com:443

:package: integration-tests/rest-client-reactive

Failed to execute goal uk.co.automatictester:truststore-maven-plugin:3.0.0:generate-truststore (wrong-host-truststore) on project quarkus-integration-test-rest-client-reactive: Execution wrong-host-truststore of goal uk.co.automatictester:truststore-maven-plugin:3.0.0:generate-truststore failed: Unable to establish TLS connection with: wrong.host.badssl.com:443


Flaky tests - Develocity

:gear: JVM Tests - JDK 21

:package: integration-tests/rest-client-reactive

io.quarkus.it.rest.client.wronghost.ExternalWrongHostTestCase.restClientRejected - History

  • 1 expectation failed. Response body doesn't match expectation. Expected: a string containing "SSLHandshakeException" Actual: ConnectTimeoutException - java.lang.AssertionError
java.lang.AssertionError: 
1 expectation failed.
Response body doesn't match expectation.
Expected: a string containing "SSLHandshakeException"
  Actual: ConnectTimeoutException

	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
  • 1 expectation failed. Response body doesn't match expectation. Expected: a string containing "SSLHandshakeException" Actual: ConnectTimeoutException - java.lang.AssertionError
java.lang.AssertionError: 
1 expectation failed.
Response body doesn't match expectation.
Expected: a string containing "SSLHandshakeException"
  Actual: ConnectTimeoutException

	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)

quarkus-bot[bot] avatar May 10 '24 23:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 347020497ad261de4d79546cd31d4286439271e0.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Initial JDK 17 Build Build Failures Logs Raw logs :mag:

You can consult the Develocity build scans.

Failures

:gear: Initial JDK 17 Build #

- Failing: extensions/vertx-http/deployment 
! Skipped: devtools/bom-descriptor-json devtools/cli docs and 412 more

:package: extensions/vertx-http/deployment

Failed to execute goal net.revelc.code:impsort-maven-plugin:1.9.0:check (check-imports) on project quarkus-vertx-http-deployment: Imports are not sorted in /home/runner/work/quarkus/quarkus/extensions/vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/StaticResourcesProcessor.java

quarkus-bot[bot] avatar May 23 '24 07:05 quarkus-bot[bot]

@mcruzdev could you fix CI?

ia3andy avatar May 26 '24 13:05 ia3andy

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 41cd583067de0157a9c303798d9f60eb812f4d02.

:white_check_mark: The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.


Flaky tests - Develocity

:gear: JVM Tests - JDK 17

:package: integration-tests/reactive-messaging-kafka

io.quarkus.it.kafka.KafkaConnectorTest.testFruits - History

  • Assertion condition defined as a Lambda expression in io.quarkus.it.kafka.KafkaConnectorTest expected: <6> but was: <5> within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a Lambda expression in io.quarkus.it.kafka.KafkaConnectorTest expected: <6> but was: <5> within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
	at io.quarkus.it.kafka.KafkaConnectorTest.testFruits(KafkaConnectorTest.java:63)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)

:gear: JVM Tests - JDK 21

:package: extensions/smallrye-reactive-messaging-kafka/deployment

io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

  • Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

quarkus-bot[bot] avatar May 27 '24 14:05 quarkus-bot[bot]

Folks, I will let you decide if it's ready to get merged and merge it if so. Thanks.

gsmet avatar Jun 07 '24 13:06 gsmet

LGTM. Nice tests. Some remarks, but not blocking.

Perhaps the missing bit is to mention this in https://quarkus.io/guides/writing-extensions but I looked briefly where it could fit, and found no mention of generating static resources. So, perhaps not required.

Thank you @FroMage, I will solve the comments soon!

mcruzdev avatar Jun 07 '24 13:06 mcruzdev

@mcruzdev it's merged, provide PRs for the comments and stuff, I will work on re-adding AdditionalStaticResources to make sure we don't break existing dev-modes and adding Path support.

ia3andy avatar Jun 07 '24 13:06 ia3andy

@mcruzdev Congratulations! 🎉

ia3andy avatar Jun 07 '24 14:06 ia3andy

Thank you for all support @ia3andy!

mcruzdev avatar Jun 07 '24 14:06 mcruzdev