Refinement when -XX:+JITServerLocalSyncCompiles is enabled by default

[mpirvu]

-XX:+JITServerLocalSyncCompiles is a JITServer specific option that allows the client JVM to perform synchronous compilations locally, at cold opt level, and follow up with remote recompilations at higher opt levels. This option has recently become the default, which makes it easier for customers, but the question is what should we do for automated testing. I argue that for most of the tests we should have this feature enabled, because we should test what we ship. However, for tests that use count=0 or disableAsyncCompilation we should always perform the synchronous compilations remotely, to stress the remote compilation aspect.

[mpirvu]

This existing method is very handy at covering both cases specified above (count=0 and disableAsyncCompilation)

bool TR::CompilationInfo::asynchronousCompilation()
   {
   static bool answer = (!TR::Options::getJITCmdLineOptions()->getOption(TR_DisableAsyncCompilation) &&
                        TR::Options::getJITCmdLineOptions()->getInitialBCount() &&
                        TR::Options::getJITCmdLineOptions()->getInitialCount() &&
                        TR::Options::getAOTCmdLineOptions()->getInitialSCount() &&
                        TR::Options::getAOTCmdLineOptions()->getInitialBCount() &&
                        TR::Options::getAOTCmdLineOptions()->getInitialCount()
                        );
   return answer;
   }

Thus we can use compInfo->asynchronousCompilation() as a test.

[mpirvu]

Currently, we set localSyncCompiles in preProcess() which may be too early for the other options to have been parsed. We need to move this code in fePostProcess() when we know about FSD. At that point, we look to see if the user has specified -XX:[+/-]JITServerLocalSyncCompiles on the command line, and if so, we'll obey. Otherwise we enable this option if compInfo->asynchronousCompilation() return false or if FSD mode.

[mpirvu]

@cjjdespres Could you please implement this feature? Thanks.

[cjjdespres]

FSD is set in feLatePostProcess, I think. Is that called before the fePostProcess functions are?

[mpirvu]

LatePostProcess comes after PostProcess

[mpirvu]

From some tracing I did a long time ago:

This is the order seen in tr.open (Java9)
processOptionsAOT creates _aotCmdLineOptions=0000000002CC38D0
  fePreProcess this=0000000002CC38D0
  jitPreProcess this=0000000002CC38D0
  processOptions  for 0000000002CC38D0
     jitPostProcess this=0000000002CC38D0
     fePostProcessAOT this=0000000002CC38D0
processOptionsJIT creates _cmdLineOptions=0000000002CC3D50
   fePreProcess this=0000000002CC3D50
   jitPreProcess this=0000000002CC3D50
   processOptions  for 0000000002CC3D50
      jitPostProcess this=0000000002CC3D50
      fePostProcessJIT this=0000000002CC3D50
feLatePostProcess this=0000000002CC38D0  AOT
   setCounts()
feLatePostProcess this=0000000002CC3D50  JIT
   setCounts()