dffml icon indicating copy to clipboard operation
dffml copied to clipboard

df: kubernetes: job: Kubernetes based orchestrator

Open johnandersen777 opened this issue 4 years ago • 2 comments

Orchestrator for running dataflows in kubernetes using individual jobs.

johnandersen777 avatar Oct 09 '21 13:10 johnandersen777

In progress

johnandersen777 avatar Oct 09 '21 13:10 johnandersen777

Kubernetes has a system for strong workload identity. All workloads are associated with service accounts, and they have short-lived OpenID-Connect (OIDC) identity-tokens issued by Kubernetes. The Kubernetes API server signs these OIDC tokens, and other workloads can validate tokens through the Kubernetes API server. This provides strong identities for workloads running on Kubernetes and can be used as a foundation for role-based authentication and authorization.

Instead of using Kubernetes Secrets, developers should base authentication and authorization on OIDC tokens. This means that instead of, e.g., storing a database password in a Secret resource, we should ensure that our database only accepts requests when presented with a valid, unexpired token.

Examples of OIDC token usage to integrate with external systems are AWS IAM roles for service accounts and Hashicorp Vault Kubernetes auth.

Source: https://www.eficode.com/blog/the-future-of-kubernetes-and-why-developers-should-look-beyond-kubernetes-in-2022

johnandersen777 avatar Mar 13 '22 15:03 johnandersen777