incubator-livy icon indicating copy to clipboard operation
incubator-livy copied to clipboard

[LIVY-865][SERVER] Fix bug that Livy identifies valid yarn state of batch session as FAILED

Open WinkerDu opened this issue 2 years ago • 3 comments

What changes were proposed in this pull request?

yarnClient.getApplicationReport(appId) would return 2 states below:

  1. YarnApplicationState (short as yarnState)
  2. FinalApplicationStatus (short as finalState)

These 2 states may switch sequentially in practice, means (yarnState, finalState) as (RUNNING, SUCCEEDED) would switch to (FINISHED, SUCCEEDED) finally. For now, (yarnState, finalState) as (RUNNING, SUCCEEDED) would be identified as FAILED state combination

This PR introduce new logic to map yarn state combination as bellow:

  1. yarnState as NEW、NEW_SAVING、SUBMITTED、ACCEPTED would be identified as STARTING
  2. yarnState as RUNNING would be identified as RUNNING
  3. other yarnState and finalState combinations would be identified to rules:
  • (FAILED, FAILED) -> FAILED
  • (KILLED, KILLED) -> KILLED
  • (FINISHED, SUCCEEDED) -> FINISHED
  • any other combination -> FAILED

How was this patch tested?

added UT.

WinkerDu avatar Aug 01 '21 13:08 WinkerDu

@jerryshao @mgaido91 would you please help to review, thanks!

WinkerDu avatar Aug 01 '21 14:08 WinkerDu

I am not familiar with this part, sorry

mgaido91 avatar Aug 02 '21 13:08 mgaido91

I am not familiar with this part, sorry

@mgaido91 thx for the reply, can you introduce any reviewer familiar with?

WinkerDu avatar Aug 02 '21 13:08 WinkerDu