soopervisor icon indicating copy to clipboard operation
soopervisor copied to clipboard

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.

Results 24 soopervisor issues
Sort by recently updated
recently updated
newest added

We want to add support for pipelines exported to argo and kubeflow to request specific resources for a given task (memory, CPU, GPU). I'm thinking we could have a mapping...

one that does not use the sample slurm cluster

[the current example](https://soopervisor.readthedocs.io/en/latest/api/kubernetes.html) works for local cluster but won't work for a production one, it needs to be a PVC ([see an example here](https://soopervisor.readthedocs.io/en/latest/tutorials/kubernetes.html#google-cloud)) we need to ensure that this...

I tried running the example (using KubernetesPodOperator) and it failed, something in Airflow broke. I tried downgrading Airflow, the [k8s provider package](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/), and the [k8s package](https://pypi.org/project/kubernetes/), but didn't find the...