Introduce profile-guided optimization to APM Server

[1pkg]

Context

Go compiler added support for profile-guided optimization (PGO) since Go 1.20, thus allowing to build further optimized Go binaries using CPU profiles. Depending on the workload, optimizations and the profile quality one can expect to extract somewhere from 2-14% of performance gains according to go.dev/blog which is quite meaningful. Further more for CPU bound services, in general, the expected results should be somewhere in the upper bound of the expected performance gains.

Requested Changes

We should capture this performance improvement in APM Server.

We should consider how we will collect the profiles, initially we could leverage existing benchmarks workflow to collect CPU profiles which will be included into the future PGO builds.

We should consider how we will store profiles and include them into the builds, the simplest and most convenient approach to commit them directly in the source code for easy distribution and repeatable builds.

Additional Links

go.dev/doc/pgo go.dev/blog/pgo