Add basic Develocity integration and enable build caching

[ribafish]

As one of the impactful Eclipse projects, we (the Develocity Solutions team) would like to invite you to be a part of the Eclipse Develocity evaluation initiative.

Description

This PR publishes a build scan for every CI build and for every local build from an authenticated Eclipse committer. The build will not fail if publishing fails. Setting the access key env var was added in relevant CI pipelines, however, the secret will need to be set by Eclipse infra - you can initiate that by opening a helpdesk ticket.

The build scans of the Eclipse Collections project are published to the Develocity instance at develocity-staging.eclipse.org, hosted by the Eclipse Foundation and run in partnership between the Eclipse and Gradle. This Develocity instance has all features and extensions enabled and is freely available for use by the Eclipse Collections project and all other Eclipse projects.

On this Develocity instance, the Eclipse Collections project will have access not only to all of the published build scans but also to other aggregate data features such as:

• Dashboards to view all historical build scans, along with performance trends over time. For example, look at the Quarkus instance trends dashboard with the ci filter applied.
• Build failure analytics for enhanced investigation and diagnosis of build failures. You can also explore the Quarkus instance for example of a different OSS projects Develocity instance.
• Test failure analytics to better understand trends and causes around slow, failing, and flaky tests. Going to the Quarkus instance again, this will probably give you a better representation of the type of data available. You can also order it by flakiness, which is a good starting point when deciding which tests need fixing the most.

This will also enable you to (optionally) use build time optimization features, such as (remote) build caching and Predictive Test Selection. In this PR, build caching is enabled, with the local cache being disabled on CI and only allowing CI to write to the remote cache, as per the general recommendations.

I ran some tests about build speed improvements when caching is configured with the following results. Here are the best-case (no code change) savings:

Invocation	Build time without cache	Build time with cache	Build time savings
install -DskipTests=true	2m 46.859s	26.396s	2m 20s (84%)
install	5m 5.385s	2m 45.928s	2m 19.457s (46%)
verify checkstyle:check --activate-profiles all -DskipTests=true	4m 11.061s	26.894s	3m 44.167s (89%)
install --projects performance-tests --also-make --activate-profiles all -DskipTests=true	1m 8.484s	15.138s	53.346s (77%)

There are additional savings to be made when running install goal if the runOrderRandomSeed would be fixed, as that is an unstable goal input, because of which the cache key calculated differs, thus prevent the cache to get hits.

Caching can be tested without setting up the Develocity credentials - simply run 2 or more builds (the first one will fill the local cache, next ones can use it).

More information can be read in the Eclipse announcement. Here is also a blog post abut Develocity from Eclipse Foundation, as well as a Develocity presentation, made in collaboration with Eclipse Foundation (and Gradle).

Please let me know if there are any questions about the value of Develocity or the changes in this pull request and I’d be happy to address them.

IMPORTANT

To get scans publishing on CI, a helpdesk ticket needs to be opened in order for those credentials to be created, as explained in the initiative documentation.