Load the existing pods when initializing kubernetes client to cleanup terminated app pods

[turboFei]

Why are the changes needed?

To prevent the terminated app pods leak if the events missed during kyuubi server restart.

How was this patch tested?

Manual test.

:2025-06-17 17:50:37.275 INFO [main] org.apache.kyuubi.engine.KubernetesApplicationOperation: [KubernetesInfo(Some(28),Some(dls-prod))] Found existing pod kyuubi-xb406fc5-7b0b-4fdf-8531-929ed2ae250d-8998-5b406fc5-7b0b-4fdf-8531-929ed2ae250d-8998-90c0b328-930f-11ed-a1eb-0242ac120002-0-20250423211008-grectg-stm-17da59fe-caf4-41e4-a12f-6c1ed9a293f9-driver with label: kyuubi-unique-tag=17da59fe-caf4-41e4-a12f-6c1ed9a293f9 in app state FINISHED, marking it as terminated
2025-06-17 17:50:37.278 INFO [main] org.apache.kyuubi.engine.KubernetesApplicationOperation: [KubernetesInfo(Some(28),Some(dls-prod))] Found existing pod kyuubi-xb406fc5-7b0b-4fdf-8531-929ed2ae250d-8998-5b406fc5-7b0b-4fdf-8531-929ed2ae250d-8998-90c0b328-930f-11ed-a1eb-0242ac120002-0-20250423212011-gpdtsi-stm-6a23000f-10be-4a42-ae62-4fa2da8fac07-driver with label: kyuubi-unique-tag=6a23000f-10be-4a42-ae62-4fa2da8fac07 in app state FINISHED, marking it as terminated

The pods are cleaned up eventually.

Was this patch authored or co-authored using generative AI tooling?

No.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 30 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (7fbeea6) to head (7f76cf5). Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
...kyuubi/engine/KubernetesApplicationOperation.scala	0.00%	30 Missing :warning:

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #7101   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         697     697           
  Lines       43214   43243   +29     
  Branches     5855    5859    +4     
======================================
- Misses      43214   43243   +29     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[turboFei]

cc @pan3793

[turboFei]

How about now? @pan3793

Due now we always cleanup terminated pods, so I think it is not necessary to involve a new config to cleanup them on kubernetes client init.

[pan3793]

my major concern is that something might go wrong during cleanTerminatedAppPodsOnKubernetesClientInitialize, but now it runs in async, failure won't block the original procedure, config is not required then.

[turboFei]

thanks, merged to main and branch-1.10

[pan3793]

@turboFei branch-1.10 broken after merging this PR

Error: ] /home/runner/work/kyuubi/kyuubi/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala:110: too many arguments (3) for method markApplicationTerminated: (pod: io.fabric8.kubernetes.api.model.Pod, eventType: org.apache.kyuubi.engine.KubernetesResourceEventTypes.KubernetesResourceEventType)Unit