Add app state to AppInfo

[AFFogarty]

What changes were proposed in this pull request?

This PR adds app state to AppInfo. This change gives clients more power to understand the state of their application and to debug any failures.

For example, there is currently no way to differentiate between interactive sessions that have gone into YARN state FAILED or YARN state FINISHED. Livy reports both of these as dead. With this new change, a client could inspect the appState to see the true state of the YARN application.

How was this patch tested?

This change includes unit tests.

CC: @rapoth @imback82

[codecov-commenter]

Codecov Report

Merging #303 into master will increase coverage by 0.02%. The diff coverage is 90.00%.

@@             Coverage Diff              @@
##             master     #303      +/-   ##
============================================
+ Coverage     68.48%   68.51%   +0.02%     
- Complexity      840      841       +1     
============================================
  Files           103      103              
  Lines          5940     5948       +8     
  Branches        898      898              
============================================
+ Hits           4068     4075       +7     
- Misses         1312     1314       +2     
+ Partials        560      559       -1     

Impacted Files	Coverage Δ	Complexity Δ
...rc/main/scala/org/apache/livy/utils/SparkApp.scala	80.64% <87.50%> (+4.64%)	1.00 <0.00> (ø)
...la/org/apache/livy/server/batch/BatchSession.scala	86.86% <100.00%> (+0.13%)	14.00 <0.00> (ø)
...e/livy/server/interactive/InteractiveSession.scala	69.85% <100.00%> (+0.08%)	51.00 <0.00> (ø)
...cala/org/apache/livy/scalaapi/ScalaJobHandle.scala	62.79% <0.00%> (-2.33%)	0.00% <0.00%> (ø%)
...ain/scala/org/apache/livy/utils/SparkYarnApp.scala	73.75% <0.00%> (-1.25%)	40.00% <0.00%> (ø%)
.../scala/org/apache/livy/sessions/SessionState.scala	59.45% <0.00%> (ø)	0.00% <0.00%> (ø%)
...ain/java/org/apache/livy/rsc/driver/RSCDriver.java	80.83% <0.00%> (+0.83%)	45.00% <0.00%> (ø%)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 97cf2f7...4edc259. Read the comment docs.

[imback82]

@jerryshao Could you help reviewing this PR? Thanks in advance!

[AFFogarty]

Not sure why an unrelated test in livy-rsc failed this time. Can we re-run the build?

[andrasbeni]

@AFFogarty, I'm curious what you think of the following alternatives:

• Mapping SparkApp.State.FAILED to SessionState.ERROR()
• Introducing SessionState.FAILED()

I'm not too familiar with the state transitions, so above ideas may be dumb. In which case I'm happy to learn why.

Other than that, I think you should create a jira issue and add its ID to the commit message

[rapoth]

@jerryshao Any chance you could review this PR? Thanks a lot for your time in advance!