native-build-tools icon indicating copy to clipboard operation
native-build-tools copied to clipboard

Add support for PGO

Open melix opened this issue 2 years ago • 1 comments

Now that GraalVM 23 is out with PGO support available in the Oracle GraalVM distribution, it would be good if the Gradle and Maven plugins added built-in support for PGO.

For the Gradle plugin, my recommendation would be to add a couple of options to org.graalvm.buildtools.gradle.dsl.NativeImageCompileOptions:

  • a Property<Boolean> getPgo() which can also be enabled via a CLI flag (e.g --pgo), which would enable the compilation with --pgo-instrument (and potentially add a -instrumented suffix to the image name for clarity), which would be used for the 1st step, building an image with instrumentation support
  • a DirectoryProperty getPgoProfilesDirectory() pointing to a directory of .iprof files which would be automatically added to the native image compilation if getPgo() returns false (and that there are actually files there)

We could have a conventional location of those profile files, for example src/native-image/<image name>/pgo, so for example for the main image it would be src/native-image/main/pgo (with the idea that these files would be checked in VCS).

This is not about the ability to automatically run the instrumented image with a "load test" and automating the process from A-Z, but at least to make it possible to enable PGO in a standard way.

melix avatar Jun 20 '23 16:06 melix

Thanks for the suggestions, @melix. I've got some comments:

a CLI flag (e.g --pgo), which would enable the compilation with --pgo-instrument

I'd like a CLI flag, but using --pgo for --pgo-instrument is probably very confusing, considering there's also a --pgo in Native Image itself. I'd still vote for --pgo-instrument even though it's longer.

and potentially add a -instrumented suffix to the image name for clarity

yes, that'd be very useful.

a DirectoryProperty getPgoProfilesDirectory() pointing to a directory of .iprof files which would be automatically added to the native image compilation if ...

Does this mean that Native Image will automatically pick up the profile stored in a default.iprof if it is in that directory? What if there are multiple profiles? Maybe users will think Native Image picks up all profiles (native-image-configure supports merging multiple profiles for example), while it really only picks up the default.iprof? Or should/could the plugin run the merge tool and feed the result into Native Image?

for the main image it would be src/native-image/main/pgo

maybe pgo-profiles or pgo-data? just an idea :)

This is not about the ability to automatically run the instrumented image.

I'm actually wondering how the iprof files find their way into this new directory for PGO profiles? We probably should provide users with the right /path/to/image-instrumented -XX:ProfilesDumpFile=full/path/to/src/native-image/main/pgo command and instructions to make sure the profiles end up in the right directory?

This is not about the ability to automatically run the instrumented image.

Quoting this a second time for another thought: why is this not about the ability to automatically run the instrumented image? I mean the plugins already support running with the tracing agent, couldn't it similarly run the instrumented binary, add the right -XX:ProfilesDumpFile flag, and tell the user to kill the app when done?

fniephaus avatar Jun 26 '23 07:06 fniephaus