k8s-spark-scheduler
k8s-spark-scheduler copied to clipboard
palantir
A Kubernetes Scheduler Extender to provide gang scheduling support for Spark on Kubernetes
###### _excavator_ is a bot for automating changes across repositories. Changes produced by the go/godel-update-plugins-oss check. To enable or disable this check, please contact the maintainers of Excavator.
###### _excavator_ is a bot for automating changes across repositories. Changes produced by the circleci/manage-circleci-oss check. To enable or disable this check, please contact the maintainers of Excavator.
###### _excavator_ is a bot for automating changes across repositories. Changes produced by the excavator/policy-bot-oss check. To enable or disable this check, please contact the maintainers of Excavator.
The conversion webhook for resource reservations now runs standalone in a separate process, so we can stop serving the webhook from the scheduler process.
This PR adds a resource reservation migrator which converts the stored version of all resource reservation objects to v1beta2. Questions: * Is there a better place to write the migration...
related to https://github.com/palantir/k8s-spark-scheduler/issues/152#issuecomment-877627500 As I did not include the overhead when setting memory annotations, the scheduler tries to assign pods to nodes which have no enough (memory) resources. Also, I...
###### _excavator_ is a bot for automating changes across repositories. Changes produced by the go/godel-update-version-oss check. To enable or disable this check, please contact the maintainers of Excavator.
###### _excavator_ is a bot for automating changes across repositories. Changes produced by the go/manage-go-mod-dependencies-oss check. go mod operation failed. This may mean that there are legitimate dependency issues with...
when i submit a batch of spark jobs, it runs doesn't like the expection. Some executor pods stucking although there are enough resources in each node for it to run....
Hi I'm trying to figure out if the k8s-spark-scheduler work in conjunction with the cluster-autoscaler. My scenario is that I'm trying to have a node-pool that is small, but when...
