bazel-gazelle icon indicating copy to clipboard operation
bazel-gazelle copied to clipboard

gazelle: add CPU profiling to aid troubleshooting

Open sluongng opened this issue 2 years ago • 3 comments

As Gazelle becomes the defacto tool to generate BUILD files for several languages in Bazel ecosystem, we gain more contributors and extension authors from a diverse set of background and experience.

However, Gazelle itself is complicated and not all the contributors are familiar with the Go ecosystem. This led to difficulty in developing language extensions and troubleshooting edge cases.

Provides a way to turn on Go profiling(1) is a great way to help with that. Go's pprof tool is well-documented and CLI friendly. Rules author could simply turn on the CPU profile capture to analyze it and figure out where is the bottleneck, how does the call stack look like.

Add environment variables GAZELLE_CPU_PROFILE to collect Go CPU profile for later analysis.

(1): https://go.dev/blog/pprof

sluongng avatar Feb 12 '23 12:02 sluongng

Note: This PR is mostly to start a discussion on what the interface to Gazelle's profiling should look like. I think an environment variable is better than a flag as it's a development feature only.

I also intend to add more documentation on how to use this profiling feature before it's merged. But would love to hear from current maintainers about the direction of the PR.

sluongng avatar Feb 12 '23 12:02 sluongng

image image

Some examples of the goodies of having this feature. I believe these visualization this shoulds help Extension Author to learn / onboard Gazelle a lot easier.

sluongng avatar Feb 12 '23 13:02 sluongng

@sluongng @achew22 do you have any updates on this? I would love to use the feature! Any way I can help?

alexbozhenko avatar Aug 23 '23 21:08 alexbozhenko

@sluongng I notice this is similar to: https://github.com/bazelbuild/bazel-gazelle/pull/1685

I wonder if we should extract this outside of fix-update like you have it.

tyler-french avatar Apr 12 '24 14:04 tyler-french

@tyler-french yeah I think having it on main() would give you the most complete picture.

I recently sent https://github.com/bazelbuild/bazel-gazelle/pull/1766 as well, which added tracing in addition to profiling, using the recent 1.22 flight recorder package.

I will close this PR though, since yours has already landed. If anyone else wants to recreate this on top of #1685, please feel free to do so.

sluongng avatar Apr 12 '24 14:04 sluongng

@tyler-french yeah I think having it on main() would give you the most complete picture.

I recently sent #1766 as well, which added tracing in addition to profiling, using the recent 1.22 flight recorder package.

I will close this PR though, since yours has already landed. If anyone else wants to recreate this on top of #1685, please feel free to do so.

Sure, will do this if I have some bandwidth soon. Thanks!

tyler-french avatar Apr 12 '24 14:04 tyler-french