Develocity integration

[ribafish]

trafficstars

@gutoveronezi, it was nice meeting you at Community over Code today. This PR will enable you to publish Build Scans to ge.apache.org as discussed.

Description

This PR publishes a build scan for every CI build and for every local build from an authenticated Apache committer. The build will not fail if publishing fails. Local and remote caching was left disabled on this PR by design so that the build is not affected by this change.

The build scans of the Apache Cloudstack project are published to the Develocity instance at ge.apache.org, hosted by the Apache Software Foundation and run in partnership between the ASF and Gradle. This Develocity instance has all features and extensions enabled and is freely available for use by the Apache Cloudstack project and all other Apache projects.

On this Develocity instance, Apache Cloudstack will have access not only to all of the published build scans but other aggregate data features such as:

• Dashboards to view all historical build scans, along with performance trends over time
• Build failure analytics for enhanced investigation and diagnosis of build failures
• Test failure analytics to better understand trends and causes around slow, failing, and flaky tests

Please let me know if there are any questions about the value of Develocity or the changes in this pull request and I’d be happy to address them.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
• [x] build/CI

[boring-cyborg[bot]]

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) Here are some useful points:

• In case of a new feature add useful documentation (raise doc PR at https://github.com/apache/cloudstack-documentation)
• Be patient and persistent. It might take some time to get a review or get the final approval from the committers.
• Pay attention to the quality of your code, ensure tests are passing and your PR doesn't have conflicts.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Issues, Mailing list and Slack.
• Be sure to read the CloudStack Coding Conventions. Apache CloudStack is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] (https://cloudstack.apache.org/mailing-lists.html) Slack: https://apachecloudstack.slack.com/

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 17.56%. Comparing base (86ae1fe) to head (2f8416d). :warning: Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #9167   +/-   ##
=========================================
  Coverage     17.56%   17.56%           
+ Complexity    15539    15537    -2     
=========================================
  Files          5911     5911           
  Lines        529359   529359           
  Branches      64655    64655           
=========================================
+ Hits          92979    92982    +3     
+ Misses       425920   425919    -1     
+ Partials      10460    10458    -2     

Flag	Coverage Δ
uitests	3.58% <ø> (ø)
unittests	18.63% <ø> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[GutoVeronezi]

@ribafish, thanks for the PR; it was nice to meet you at the conference. Now we will run the tests and, as soon we meet the requirements, we will merge the PR.

[GutoVeronezi]

@blueorangutan package

[blueorangutan]

@GutoVeronezi a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9793

[DaanHoogland]

@GutoVeronezi what is your intention with this? Are you proposing to base some policy on this?

Also, do we need more testing than the github actions and build/packaging?

[GutoVeronezi]

@GutoVeronezi what is your intention with this? Are you proposing to base some policy on this?

The main goal is to identify slow and flaky tests, so we can improve our code and workflow. Having those metrics can also help us to identify other problems that might be there and we are not seeing.

Also, do we need more testing than the github actions and build/packaging?

Not sure if more, but well structured.

[DaanHoogland]

Also, do we need more testing than the github actions and build/packaging?

Not sure if more, but well structured.

@GutoVeronezi , my communication skill are way below par. I meant to ask if we need more testing on this PR. You are absolutely right in your reply and it is valid with repect to the question I posed, just not to the one I intended ;)

[GutoVeronezi]

Also, do we need more testing than the github actions and build/packaging?

Not sure if more, but well structured.

@GutoVeronezi , my communication skill are way below par. I meant to ask if we need more testing on this PR. You are absolutely right in your reply and it is valid with repect to the question I posed, just not to the one I intended ;)

I see hehe

It should not affect the system itself; however, I would run it just for sanity.

[vishesh92]

@ribafish Thanks for PR.

I don't remember exactly, but there was some issue related to security in github actions resulting in leaking of secrets. One of the reasons we disabled sonar on PRs from forks. @DaanHoogland Do you think that might be an issue here as well?

[weizhouapache]

@ribafish Thanks for PR.

I don't remember exactly, but there was some issue related to security in github actions resulting in leaking of secrets. One of the reasons we disabled sonar on PRs from forks. @DaanHoogland Do you think that might be an issue here as well?

It was because of target in the github actions. I think this pr doesn't have the issue. I have no idea what this pr is used for. We do not use gradle at all. Let's see

[ribafish]

Hi @vishesh92 and @weizhouapache

This PR enables you to push build scans to Apache Develocity at ge.apache.org. These capture metadata about your build, which can then be used to monitor and improve the build performance, test times and fix flaky tests. It is also a prerequisite for other build performance improvements, such as using local and remote build cache, test distribution or predictive test selection (not part of this PR).

Develocity supports multiple build tools; there are also several Apache projects built with Maven that use it already, such as Camel, Pulsar, Creadur RAT, etc. -> you can see their build scans at ge.apache.org with the Maven filter applied.

The information captured includes the execution plan and timings, tests executed, dependencies and their versions, etc., but it does not include information such as the full list of env vars or the full cli command used to start a build unless added explicitly via custom values.

[weizhouapache]

Hi @vishesh92 and @weizhouapache

This PR enables you to push build scans to Apache Develocity at ge.apache.org. These capture metadata about your build, which can then be used to monitor and improve the build performance, test times and fix flaky tests. It is also a prerequisite for other build performance improvements, such as using local and remote build cache, test distribution or predictive test selection (not part of this PR).

Develocity supports multiple build tools; there are also several Apache projects built with Maven that use it already, such as Camel, Pulsar, Creadur RAT, etc. -> you can see their build scans at ge.apache.org with the Maven filter applied.

The information captured includes the execution plan and timings, tests executed, dependencies and their versions, etc., but it does not include information such as the full list of env vars or the full cli command used to start a build unless added explicitly via custom values.

@ribafish thanks for the information as I understand, with this pr, the info of github actions will be uploaded to ge.apache.org for analysis, right ?

do we need to ask ASF infra to add the secret GE_ACCESS_TOKEN ?

[ribafish]

It is set up that any authenticated build will get published - at the moment, only CI jobs are authenticated and ASF infra already added GE_ACCESS_TOKEN to the organization secrets, so there's no need to do anything else. If you wanted to also publish local build scans, all you need to do is to authenticate with ge.apache.org, which you can do like this. To sign in to ge.apache.org you can use your ASF credentials, as your LDAP is set up with Develocity.

[weizhouapache]

It is set up that any authenticated build will get published - at the moment, only CI jobs are authenticated and ASF infra already added GE_ACCESS_TOKEN to the organization secrets, so there's no need to do anything else. If you wanted to also publish local build scans, all you need to do is to authenticate with ge.apache.org, which you can do like this. To sign in to ge.apache.org you can use your ASF credentials, as your LDAP is set up with Develocity.

cool, thanks @ribafish

I've got an access key via command mvn com.gradle:develocity-maven-extension:provision-access-key

[weizhouapache]

@ribafish quick question, is it possible to remove some build scans ?

[clayburn]

@weizhouapache - an admin can do this. Are you referring to the 4 local build scans from this project? If so, I can ensure this happens

[weizhouapache]

@weizhouapache - an admin can do this. Are you referring to the 4 local build scans from this project? If so, I can ensure this happens

@clayburn Yes, can you please remove them? thanks

[clayburn]

Done

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9874

[clayburn]

have we (a) added the token secrets.GE_ACCESS_TOKEN to the repo

GE_ACCESS_TOKEN is an org-wide secret for the GitHub Apache org, managed by ASF Infra. As such, it is available to the cloudstack repo (but not for builds originating from forks).

(b) does having this in the .mvn affect private builds who may not want to publish build related data to the central server?

No, there are two scenarios here:

• For community contributors - they will not be able to authenticate to ge.apache.org, so no data is published
• For ASF committers - no data will be shared unless the committer explicitly opts-in by provisioning an access key using ASF LDAP credentials. Otherwise, they will have unauthenticated builds, same as community contributors, so no build data will be published.

[weizhouapache]

• For ASF committers - no data will be shared unless the committer explicitly opts-in by provisioning an access key using ASF LDAP credentials. Otherwise, they will have unauthenticated builds, same as community contributors, so no build data will be published.

@clayburn it would be good to allow ASF committers to remove their own build data

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.