storm icon indicating copy to clipboard operation
storm copied to clipboard

[STORM-3683] Check if JVM options used for launching worker are valid.

Open bipinprasad opened this issue 4 years ago • 3 comments

What is the purpose of the change

Error in JVM options can cause the worker launch to fail. This change will detect the erroneous options to be detected early when topology is submitted.

How was the change tested

With new unit tests

bipinprasad avatar Aug 04 '20 18:08 bipinprasad

close/reopen for rebuild

bipinprasad avatar Aug 12 '20 14:08 bipinprasad

I think a good reason to catching this early (i.e. at submission time) is that the cycle time for error detection is longer. Case in point is JVM options changes and the debug cycle yesterday just to discover the problem with the profiler options.

bipinprasad avatar Sep 03 '20 19:09 bipinprasad

I think a good reason to catching this early (i.e. at submission time) is that the cycle time for error detection is longer.

It is not hard to find out that jvm didn't launch because of jvm option issue, GC option, profiler option or other options, since they are in the logs.

But if this feature is really desired, I would suggest to move to Nimbus code. Doing it in Nimbus.submitTopologyWithOpts is still at submission time and users will see the exception from console log. But doing it on StormSubmitter side won't be able to detect the issue because it doesn't use daemonConf.

Another advantage of doing it at Nimbus.submitTopologyWithOpts is we can avoid code duplication and we can restrict the scope of the code related to launching a worker. Because the code to launch the worker is purely server-side, it would be better to not expose it to client.

But after moving it to Nimbus code, we need to deal with nimbusConf vs supervisorConf issue (although it is a less serious problem)

Ethanlm avatar Sep 04 '20 14:09 Ethanlm