agones icon indicating copy to clipboard operation
agones copied to clipboard

Support `Passthrough` on GKE Autopilot

Open zmerlynn opened this issue 11 months ago • 0 comments

When I implemented support for GKE Autopilot (#2777), I blocked the Passthrough port policy on Autopilot because it was complicated: Right now we do a handshake with Autopilot - we create the Pod with a special annotation that says "give me a host port in range X..Y". We then backfill the GameServer object from that. That's fine for Dynamic since we can just treat Autopilot as the source of truth for the port number, but for Passthrough that doesn't work: On Passthrough we need to bind the container to the allocated port - but on Autopilot this allocation happens in an Autopilot webhook, during Pod creation. To support Passthrough, we would need a webhook on Pods after Autopilot's warden-mutating webhook that could then propagate the port setting back to the container.

Looking at webhook options, we could do this pretty safely and filter it pretty much to this exact case:

  • Let's do this all under an AutopilotPassthroughPort Feature Gate
  • When there is a Passthrough port on Autopilot, add a new label, e.g. agones.dev/port: "autopilot-passthrough"
  • In every install, add a webhook:
    • named zzz-agones-mutation-webhook: look, it's cheesy, but webhooks are run in alphabetical order (I verified this), so pick an "end of the phone book" name. But even then, we could guarantee that it's run last with a reinvocationPolicy.
    • The webhook would be scoped to Pod CREATE
    • With an objectSelector of the label we created above - this drastically limits the scope and should increase the extensions load a bit, but not nearly so much as intercepting all Pod.

The webhook could then back the container port from the allocated port.

zmerlynn avatar Mar 20 '24 00:03 zmerlynn