quarkus icon indicating copy to clipboard operation
quarkus copied to clipboard

Upload native build statistics

Open zakkak opened this issue 10 months ago • 6 comments

Since Quarkus CI already runs the native tests quite often with a fixed Mandrel version, we can gather these data and increase our insight regarding Quarkus changes that affects image build time statistics.

zakkak avatar Mar 29 '24 10:03 zakkak

Note that you could use an approach similar to:

https://github.com/quarkusio/quarkus/blob/824234e0ad13b852ba6c1666367bfa39e7b90f9c/.github/workflows/develocity-publish-build-scans.yml#L33-L39

to also add it as a step summary (which allows to visualize things right away instead of downloading the file)

gsmet avatar Apr 22 '24 13:04 gsmet

to also add it as a step summary (which allows to visualize things right away instead of downloading the file)

That would be a json file per integration-test, wouldn't it be too much noise in the report? What do you think would make more sense to be shown in the report?

zakkak avatar Apr 23 '24 12:04 zakkak


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit ec781e2e1e2091da7170dccf682175a0923745ce.

Failing Jobs

:warning: Artifacts of the workflow run were not available thus the report misses some details.

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:

quarkus-bot[bot] avatar Apr 24 '24 05:04 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 7f8549e99d2a2df887f66abba67b36163d97aeba.

Failing Jobs

:warning: Artifacts of the workflow run were not available thus the report misses some details.

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector Extract and import build stats :warning: Check → Logs Raw logs :construction:

quarkus-bot[bot] avatar Apr 24 '24 06:04 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 42d9ba9f68eac7e7893fc5901330c279e6c92a49.

Failing Jobs

:warning: Artifacts of the workflow run were not available thus the report misses some details.

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector Extract and import build stats :warning: Check → Logs Raw logs :construction:

quarkus-bot[bot] avatar Apr 24 '24 07:04 quarkus-bot[bot]

This is blocked by https://github.com/Karm/collector/issues/27

zakkak avatar Apr 24 '24 20:04 zakkak


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 42d9ba9f68eac7e7893fc5901330c279e6c92a49.

Failing Jobs

:warning: Artifacts of the workflow run were not available thus the report misses some details.

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector Extract and import build stats :warning: Check → Logs Raw logs :construction:

quarkus-bot[bot] avatar May 13 '24 13:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit c23eb7fa34abee8316d00566de7f022cb2b5ddf9.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan

quarkus-bot[bot] avatar May 27 '24 07:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 5fa39fbbfeab98eeb161b6804220b31c24694887.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan

quarkus-bot[bot] avatar May 27 '24 07:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 417953336722f73e2c0647a1498c4989dc51ba81.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector (Amazon, 80, amazon-lambda, amazon-lambda-http, amazon-lambda-res... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (AWT, ImageIO and Java2D, 30, awt, no-awt, ubuntu-latest) Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Cache, 75, infinispan-cache-jpa, infinispan-client, cache, redis... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data1, 95, jpa-h2, jpa-h2-embedded, jpa-mariadb, jpa-mssql, jpa-... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data2, 70, jpa, jpa-mapping-xml/legacy-app, jpa-mapping-xml/mode... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data3, 75, flyway, hibernate-orm-panache, hibernate-orm-panache-... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data4, 60, mongodb-client, mongodb-devservices, mongodb-panache,... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data5, 70, jpa-postgresql, jpa-postgresql-withxml, narayana-stm,... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data6, 105, elasticsearch-rest-client, elasticsearch-java-client... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data7, 85, reactive-oracle-client, reactive-mysql-client, reacti... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (DevTools Integration Tests, 75, devtools-registry-client, ubuntu... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (gRPC, 80, grpc-health, grpc-interceptors, grpc-mutual-auth, grpc... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (HTTP, 110, elytron-resteasy, resteasy-jackson, elytron-resteasy-... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Main, 45, main, ubuntu-latest) Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Messaging1, 115, kafka, kafka-ssl, kafka-sasl, kafka-avro-apicur... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Messaging2, 80, reactive-messaging-amqp, reactive-messaging-rabb... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc1, 70, maven, jackson, jsonb, kotlin-serialization, rest-cli... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc2, 70, hibernate-validator, test-extension/tests, logging-ge... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc3, 80, kubernetes-client, openshift-client, kubernetes-servi... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc4, 130, picocli-native, gradle, micrometer-mp-metrics, micro... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Security1, 65, elytron-security-oauth2, elytron-security, elytro... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Security2, 75, oidc, oidc-code-flow, oidc-tenancy, oidc-client, ... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Security3, 55, keycloak-authorization, smallrye-jwt-token-propag... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Spring, 60, spring-di, spring-web, spring-data-jpa, spring-boot-... Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Windows support, 50, resteasy-jackson, qute, windows-latest) Extract and import build stats :warning: Check → Logs Raw logs :construction:

You can consult the Develocity build scans.

quarkus-bot[bot] avatar May 27 '24 11:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 61ae566255eaeea1de3512d595ebf0c68cbf065c.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector (Amazon, 80, amazon-lambda, amazon-lambda-http, amazon-lambda-res... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (AWT, ImageIO and Java2D, 30, awt, no-awt, ubuntu-latest) Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Cache, 75, infinispan-cache-jpa, infinispan-client, cache, redis... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data1, 95, jpa-h2, jpa-h2-embedded, jpa-mariadb, jpa-mssql, jpa-... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data2, 70, jpa, jpa-mapping-xml/legacy-app, jpa-mapping-xml/mode... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data3, 75, flyway, hibernate-orm-panache, hibernate-orm-panache-... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data4, 60, mongodb-client, mongodb-devservices, mongodb-panache,... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data5, 70, jpa-postgresql, jpa-postgresql-withxml, narayana-stm,... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data6, 105, elasticsearch-rest-client, elasticsearch-java-client... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Data7, 85, reactive-oracle-client, reactive-mysql-client, reacti... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (DevTools Integration Tests, 75, devtools-registry-client, ubuntu... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (gRPC, 80, grpc-health, grpc-interceptors, grpc-mutual-auth, grpc... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (HTTP, 110, elytron-resteasy, resteasy-jackson, elytron-resteasy-... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Main, 45, main, ubuntu-latest) Extract and import build stats :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Messaging1, 115, kafka, kafka-ssl, kafka-sasl, kafka-avro-apicur... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Messaging2, 80, reactive-messaging-amqp, reactive-messaging-rabb... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc1, 70, maven, jackson, jsonb, kotlin-serialization, rest-cli... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc2, 70, hibernate-validator, test-extension/tests, logging-ge... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc3, 80, kubernetes-client, openshift-client, kubernetes-servi... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Misc4, 130, picocli-native, gradle, micrometer-mp-metrics, micro... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Security1, 65, elytron-security-oauth2, elytron-security, elytro... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Security2, 75, oidc, oidc-code-flow, oidc-tenancy, oidc-client, ... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Security3, 55, keycloak-authorization, smallrye-jwt-token-propag... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Spring, 60, spring-di, spring-web, spring-data-jpa, spring-boot-... Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:
Upload build stats to collector (Windows support, 50, resteasy-jackson, qute, windows-latest) Run actions/download-artifact@v4 :warning: Check → Logs Raw logs :construction:

You can consult the Develocity build scans.

quarkus-bot[bot] avatar May 27 '24 12:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 874d58782a17901eea98e2e2ef010156edf9084a.

Failing Jobs

Status Name Step Failures Logs Raw logs Build scan
Upload build stats to collector (Main, 45, main, ubuntu-latest) Extract and import build stats :warning: Check → Logs Raw logs :construction:

You can consult the Develocity build scans.

quarkus-bot[bot] avatar May 27 '24 12:05 quarkus-bot[bot]

This seems to work (tested with Main IT in https://github.com/zakkak/quarkus/actions/runs/9254781855)

@Karm please have a look.

zakkak avatar May 27 '24 13:05 zakkak


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 61380b833795233a33266b36b0c4ab3d129891ba.

:white_check_mark: The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.


Flaky tests - Develocity

:gear: JVM Tests - JDK 21

:package: extensions/smallrye-reactive-messaging-kafka/deployment

io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

  • Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

quarkus-bot[bot] avatar May 27 '24 14:05 quarkus-bot[bot]


:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.


Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 2512ca5d294b366bfd34db296d816dd32a9c0696.

:white_check_mark: The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.


Flaky tests - Develocity

:gear: JVM Tests - JDK 21

:package: extensions/smallrye-reactive-messaging-kafka/deployment

io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

  • Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

quarkus-bot[bot] avatar May 27 '24 17:05 quarkus-bot[bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit d470e57c3aaee547aa07b2e8dbce917ab45a2953.

:white_check_mark: The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.


Flaky tests - Develocity

:gear: JVM Tests - JDK 21

:package: extensions/smallrye-reactive-messaging-kafka/deployment

io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

  • Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

:package: extensions/smallrye-reactive-messaging/deployment

io.quarkus.smallrye.reactivemessaging.hotreload.ConnectorChangeTest.testUpdatingConnector - History

  • Expecting actual: ["-4","-5","-6","-7","-8","-9","-10","-11"] to start with: ["-3", "-4", "-5", "-6"] - java.lang.AssertionError
java.lang.AssertionError: 

Expecting actual:
  ["-4","-5","-6","-7","-8","-9","-10","-11"]
to start with:
  ["-3", "-4", "-5", "-6"]

	at io.quarkus.smallrye.reactivemessaging.hotreload.ConnectorChangeTest.testUpdatingConnector(ConnectorChangeTest.java:36)

:package: integration-tests/reactive-messaging-kafka

io.quarkus.it.kafka.KafkaConnectorTest.testFruits - History

  • Assertion condition defined as a Lambda expression in io.quarkus.it.kafka.KafkaConnectorTest expected: <6> but was: <5> within 10 seconds. - org.awaitility.core.ConditionTimeoutException
org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a Lambda expression in io.quarkus.it.kafka.KafkaConnectorTest expected: <6> but was: <5> within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
	at io.quarkus.it.kafka.KafkaConnectorTest.testFruits(KafkaConnectorTest.java:63)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)

quarkus-bot[bot] avatar May 28 '24 09:05 quarkus-bot[bot]

@zakkak The only nitpick would be some perhaps needlessly missing attributes from runner info, e.g.

Yes, we had that in place but as suggested by @jerboaa in https://github.com/quarkusio/quarkus/pull/39784#discussion_r1616184350 I agree it is better we remove them given that they don't come from the actual runner (the runner uploading the data is different than the runner that did the actual runners).

If you think we really need this info I can modify it (in a follow up PR) to use a runner-info per IT-category and report the actual numbers.

zakkak avatar May 28 '24 13:05 zakkak