[BugFix] retry get spark state from spark client log when yarn queue expire

[blanklin030]

Why I'm doing:

Yarn maintains two lists: the running tasks in list 1 and the finished tasks in list 2. To avoid excessive memory usage, 5000 lists are configured for the number of completed applications. This will cause a boundary problem. After yarn clears the appid, the asynchronous thread (load etl checker) checks the APPID after 5s. This will cause the boundary problem

What I'm doing:

After the spark task is submitted, the spark interaction log is maintained on the client. As long as the log is persisted to the disk, when the boundary problem occurs in the future, the spark log is parsed again to know the actual execution status of the task

Fixes #45694

What type of PR is this:

• [x] BugFix
• [ ] Feature
• [ ] Enhancement
• [ ] Refactor
• [ ] UT
• [ ] Doc
• [ ] Tool

Does this PR entail a change in behavior?

• [x] Yes, this PR will result in a change in behavior.
• [ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
• [ ] Parameter changes: default values, similar parameters but with different default values
• [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
• [ ] Feature removed
• [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• [x] I have added test cases for my bug fix or my new feature
• [ ] This pr needs user documentation (for new or modified features or behaviors)
  • [ ] I have added documentation for my new feature or new function
• [ ] This is a backport pr

Bugfix cherry-pick branch check:

• [x] I have checked the version labels which the pr will be auto-backported to the target branch
  • [x] 3.3
  • [x] 3.2
  • [x] 3.1
  • [x] 3.0
  • [x] 2.5

[CLAassistant]

All committers have signed the CLA.

[banmoy]

@wyb please also take a look

[github-actions[bot]]

[FE Incremental Coverage Report]

:x: fail : 23 / 37 (62.16%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
:large_blue_circle:	com/starrocks/load/loadv2/SparkEtlJobHandler.java	15	27	55.56%	[266, 267, 285, 292, 293, 294, 295, 298, 299, 300, 301, 303]
:large_blue_circle:	com/starrocks/load/loadv2/SparkLauncherMonitor.java	8	10	80.00%	[167, 168]

[sonarqubecloud[bot]]

Quality Gate passed

Issues
14 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[github-actions[bot]]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)