[Feature]: Integration with Kubernetes V2

[czy006]

Description

In #1917,The K8S already has some initial support. we will continue to enhance cloud-native capabilities in subsequent versions. Currently, Amoro has preliminary cloud-native capabilities, and we hope to strengthen this aspect in future versions. We can proceed in the following directions:

• Improve cloud-native auto-scaling functionality, allowing Opitizmer to automatically scale in different engines in Kubernetes environments.
• Enhance Helm chart-related features.
• Support high availability of AMS on ETCD.
• Improve Kubernetes-related unit test infrastructure.

Use case/motivation

No response

Describe the solution

No response

Subtasks

• [x] #1957
• [x] #1922
• [ ] #2796
• [x] #2778

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct