Support GKE AutoPilot clusters

[swithinfoote]

trafficstars

Is your feature request related to a problem? Please describe. Deployment to GKE Aotopilot clusters is not currently supported.

Describe the solution you'd like It would be great if we could deploy to our Autopilot clusters

Describe alternatives you've considered We can run a standard GKE cluster which is working fine.

Additional context

Create an Autopilot cluster in GKE and attempt to deploy. Unfortunately this fails currently.

Output from px deploy command

px deploy --kubeconfig /Users/***/.kube/config
Pixie CLI

Running Cluster Checks:
 ✔    Kernel version > 4.14.0 
 ✔    Cluster type is supported 
 ✔    K8s version > 1.12.0 
 ✔    Kubectl > 1.10.0 is present 
 ✔    User can create namespace 
 ✔    Cluster type is in list of known supported types 
Installing version: 0.7.12
Generating YAMLs for Pixie
Deploying Pixie to the following cluster: ***-autopilot
Is the cluster correct? (y/n) [y] : 
Found 5 nodes
 ✔    Creating namespace 
 ✔    Deleting stale Pixie objects, if any 
 ✔    Deploying secrets and configmaps 
 ✔    Deploying dependencies: NATS 
 ✕    Deploying Cloud Connector  ERR: admission webhook "validation.gatekeeper.sh" denied the request: [denied by autogke-no-write-mode-hostpath] hostPath volume sys used in container app uses path /sys which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: ["/var/log/"]. Requesting user: <***> and groups: <["system:authenticated"]>
FATA[0153] Failed to deploy Vizier                       error="admission webhook \"validation.gatekeeper.sh\" denied the request: [denied by autogke-no-write-mode-hostpath] hostPath volume sys used in container app uses path /sys which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [\"/var/log/\"]. Requesting user: <***> and groups: <[\"system:authenticated\"]>"

[oazizi000]

GKE Autopilot has restrictions that makes it currently incompatible with BPF:

https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview?_ga=2.142566970.-940892016.1612382046#host_options_restrictions

We need access to the host namespaces, and Autopilot does not currently allow that.

[mastersingh24]

@oazizi000 @swithinfoote - feel free to ping me directly and we can chat about how we can enable this for Autopilot.

[oazizi000]

@mastersingh24 That'd be awesome! Can you join the Pixie community slack? pixie-community.slack.com

[mastersingh24]

@mastersingh24 That'd be awesome! Can you join the Pixie community slack? pixie-community.slack.com

Done

[sourcec0de]

Looks like they published eBPF support. Not sure if the limitations originally mentioned by @oazizi000 are still present. https://cloud.google.com/blog/products/containers-kubernetes/ip-masquerading-and-ebpf-are-now-in-gke-autopilot