volcano icon indicating copy to clipboard operation
volcano copied to clipboard

Is it possible to use task topology in spark client mode?

Open MatheusNtg opened this issue 1 year ago • 2 comments

I've a cluster where I create the driver pod as a task of a job in client mode. This driver will create the executors as requested, but I have the following questions: Will these executors be scheduled as tasks? Or they will be scheduled as pods by the k8s? Also, I want that both driver and created executors reside in the same AZ so I was thinking to use task-topology to do that. However, I think that task topology will only work if both driver and executors are tasks, is that correct? This is why I want to know if in this scenario the executors created by the driver in client mode are task or not.

MatheusNtg avatar Jan 31 '24 14:01 MatheusNtg

cc @Yikun

Monokaix avatar Feb 01 '24 06:02 Monokaix

Is there any update about this?

MatheusNtg avatar Feb 20 '24 19:02 MatheusNtg