volcano icon indicating copy to clipboard operation
volcano copied to clipboard

Spark client mode doesn’t support PodGroup Template?

Open xuanlang opened this issue 1 year ago • 1 comments

Please describe your problem in detail

spark-version:3.5.2 volcano-version:1.9.0

When I submit a job using spark-sql, I always get the following error:

Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://*.*.*.*:10000/api/v1/namespaces/ns-hdf/pods. Message: admission webhook "validatepod.volcano.sh" denied the request: failed to get PodGroup for pod <ns-hdf/tpcds-10-face4e921daafccd-exec-1>: podgroups.scheduling.volcano.sh "spark-10d04fd77c9545059297dae304568d93-podgroup" not found. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "validatepod.volcano.sh" denied the request: failed to get PodGroup for pod <ns-hdf/tpcds-10-face4e921daafccd-exec-1>: podgroups.scheduling.volcano.sh "spark-10d04fd77c9545059297dae304568d93-podgroup" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=null, status=Failure, additionalProperties={}).

Any other relevant information

No response

xuanlang avatar Sep 23 '24 10:09 xuanlang

You can refer to https://spark.apache.org/docs/latest/running-on-kubernetes.html#using-volcano-as-customized-scheduler-for-spark-on-kubernetes and set --conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml flag.

Monokaix avatar Sep 24 '24 01:09 Monokaix

Hello 👋 Looks like there was no activity on this issue for last 90 days. Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity for 60 days, this issue will be closed (we can always reopen an issue if we need!).

stale[bot] avatar Feb 01 '25 01:02 stale[bot]

Closing for now as there was no activity for last 90 days after marked as stale, let us know if you need this to be reopened! 🤗

stale[bot] avatar May 06 '25 02:05 stale[bot]