che-server
che-server copied to clipboard
eclipse-che
Feature/develocity build cache
What does this PR do?
This PR will enable you to publish Build Scans to develocity-staging.eclipse.org, as well as use build caching, as explained in the issue https://github.com/eclipse-che/che/issues/23358.
Description
This PR publishes a build scan for every CI build and for every local build from an authenticated Eclipse committer. The build will not fail if publishing fails. Setting the access key env var was added in relevant Github Actions workflows, however, the secret will need to be set by Eclipse infra - you can initiate that by opening a helpdesk ticket.
The build scans of the Eclipse Che-Server project are published to the Develocity instance at develocity-staging.eclipse.org, hosted by the Eclipse Foundation and run in partnership between the Eclipse and Gradle. This Develocity instance has all features and extensions enabled and is freely available for use by the Eclipse Che-Server project and all other Eclipse projects.
On this Develocity instance, the Eclipse Che-Server project will have access not only to all of the published build scans but also to other aggregate data features such as:
- Dashboards to view all historical build scans, along with performance trends over time. For example, look at the Quarkus instance trends dashboard with the
cifilter applied. - Build failure analytics for enhanced investigation and diagnosis of build failures. You can also explore the Quarkus instance for example of a different OSS projects Develocity instance.
- Test failure analytics to better understand trends and causes around slow, failing, and flaky tests. Going to the Quarkus instance again, this will probably give you a better representation of the type of data available. You can also order it by flakiness, which is a good starting point when deciding which tests need fixing the most.
This will also enable you to (optionally) use build time optimization features, such as (remote) build caching and Predictive Test Selection. In this PR, build caching is enabled, with the local cache being disabled on CI and only allowing CI to write to the remote cache, as per the general recommendations.
More information can be read in the Eclipse announcement.
Please let me know if there are any questions about the value of Develocity or the changes in this pull request and Iād be happy to address them.
IMPORTANT
To get scans publishing on CI, a helpdesk ticket needs to be opened in order for those credentials to be created, as explained in the initiative documentation.
Screenshot/screencast of this PR
An example build scan of a local build can be observed here. You can also observe that the build time was reduced by 1m 35s by using the build cache.
I've also found that the build time could further be improved by fixing volatile goal inputs, which manifest itself as the generated DtoServerImpls.java, which is not deterministic -> ordering differs on different runs. By making the DtoGenerator generate deterministic files, the build time could be reduced by additional 2m 6s.
What issues does this PR fix or reference?
https://github.com/eclipse-che/che/issues/23358
How to test this PR?
As explained in the Eclipse Develocity documentation, you can start publishing scans locally if you run the mvn develocity:provision-access-key goal, which will take you to develocity-staging.eclipse.org, where you can log in with your eclipse account and confirm access key generation. All the subsequent builds will be published.
Caching can be tested without setting up the Develocity credentials - simply run 2 or more builds (the first one will fill the local cache, next ones can use it).
PR Checklist
As the author of this Pull Request I made sure that:
- [x] The Eclipse Contributor Agreement is valid
- [x] Code produced is complete
- [x] Code builds without errors
- [ ] Tests are covering the bugfix
- [ ] The repository devfile is up to date and works
- [x] Sections
What issues does this PR fix or referenceandHow to test this PRcompleted - [ ] Relevant user documentation updated
- [ ] Relevant contributing documentation updated
- [x] CI/CD changes implemented, documented and communicated
Release Notes
Reviewers
Reviewers, please comment how you tested the PR when approving it.
Hi @ribafish. Thanks for your PR.
I'm waiting for a eclipse-che member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
![openshift-ci[bot] avatar](https://avatars.githubusercontent.com/in/91280?v=4 "Published by a pub.dev verified publisher"){kind=small}
@ribafish thank you for the contribution, related ticket https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/5689
@SDawley @vinokurig please, review and let me know if there are any concerns
@ribafish once merged, please add the dedicated Develocity section to the README with all the relevant details and available charts / dashboards - https://github.com/eclipse-che/che-server/blob/main/README.md
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: ribafish, vinokurig
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
@vinokurig, I believe the tests failing were due to a transient issue, but since I'm not a committer, the bot doesn't obey my /retest command. Could you trigger a retest please?
looks like the /retest command does not help because the build is failing. @ribafish I will reopen the pull request from the repository branch to have the build job done.
@ribafish Could you please rebase the pull request into a single commit, so I could cherry-pick it to a branch?
@ribafish: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:
| Test name | Commit | Details | Required | Rerun command |
|---|---|---|---|---|
| ci/prow/v14-gitlab-no-pat-oauth-flow-ssh-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitlab-no-pat-oauth-flow-ssh-url |
| ci/prow/v14-azure-no-pat-oauth-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-azure-no-pat-oauth-flow |
| ci/prow/v14-gitea-no-pat-oauth-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitea-no-pat-oauth-flow |
| ci/prow/v14-bitbucket-no-pat-oauth-flow-ssh-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-bitbucket-no-pat-oauth-flow-ssh-url |
| ci/prow/v14-github-no-pat-oauth-flow-raw-devfile-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-github-no-pat-oauth-flow-raw-devfile-url |
| ci/prow/v14-gitlab-with-oauth-setup-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitlab-with-oauth-setup-flow |
| ci/prow/v14-gitlab-no-pat-oauth-flow-raw-devfile-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitlab-no-pat-oauth-flow-raw-devfile-url |
| ci/prow/v14-azure-no-pat-oauth-flow-ssh-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-azure-no-pat-oauth-flow-ssh-url |
| ci/prow/v14-azure-with-pat-setup-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-azure-with-pat-setup-flow |
| ci/prow/v14-gitea-with-pat-setup-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitea-with-pat-setup-flow |
| ci/prow/v14-github-no-pat-oauth-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-github-no-pat-oauth-flow |
| ci/prow/v14-che-smoke-test | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-che-smoke-test |
| ci/prow/v14-gitlab-with-pat-setup-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitlab-with-pat-setup-flow |
| ci/prow/v14-github-with-pat-setup-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-github-with-pat-setup-flow |
| ci/prow/v14-azure-no-pat-oauth-flow-raw-devfile-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-azure-no-pat-oauth-flow-raw-devfile-url |
| ci/prow/v14-bitbucket-no-pat-oauth-flow-raw-devfile-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-bitbucket-no-pat-oauth-flow-raw-devfile-url |
| ci/prow/v14-gitlab-no-pat-oauth-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-gitlab-no-pat-oauth-flow |
| ci/prow/v14-bitbucket-no-pat-oauth-flow | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-bitbucket-no-pat-oauth-flow |
| ci/prow/v14-github-no-pat-oauth-flow-ssh-url | d3578d89d9bda5ae678dff25125bfc7ae410e72a | link | true | /test v14-github-no-pat-oauth-flow-ssh-url |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.
I cherry-picked the commit into a new branch of the original eclipse/che-server repository a pull request from that branch to have all the build jobs done: https://github.com/eclipse-che/che-server/pull/781
We have a known issue https://github.com/eclipse-che/che/issues/23053 about pull request build failure. Since the build check of my pull request with the commit from this pull request passed, I am going to merge this pr after 7.100 release.
Merged the https://github.com/eclipse-che/che-server/pull/781 pr as this pr can not be merged due to the build failure.