Add support for gcp.compute.Firewall, gcp.compute.RouterNat and gcp.compute.Router

[tricky42]

What problem are you facing?

We cant to switch from Pulumi to Crossplane for managing our infrastructure. So far we can switch fully as not all needed resources are supported by the Crossplane GCP provider:

• gcp.compute.Firewall
• gcp.compute.RouterNat
• gcp.compute.Router

We are using private GKE clusters and use Router and RouterNat to enable access to DockerHub until we have proxy the needed images in our own registry. We configure Firewall to allow Master nodes to access resources which are deployed (e.g. Admission Webhooks (Prometheus) or accessing Kubeseal Service using CLI.

How could Crossplane help solve your problem?

Supporting these resources would allow us to switch to Crossplane.

[negz]

Thanks for the feature requests @tricky42! I'll move this over to provider-gcp, where these features would be implemented.

[psinghal20]

Hey @negz, if no one is actually working on these resources, I would love to take this issue up. I have started with the Firewall resource implementation.

TBS stream on Pub/Sub implementation by @muvaf @hasheddan is an extremely helpful starting point! Thanks for such a great resource! :smile:

[sergenyalcin]

Hi all, while looking this issue, I noticed a few things that I would like to share with you.

Firstly (to understand the api of the resources), I started to look the API document of the Router REST resource and I observed a NAT array in the fields list. From this point, I started to investigate if NAT is a separate REST resource.

When I looked at NAT's reference document (https://cloud.google.com/nat/docs/apis) I saw the following statement: "Cloud NAT is configured using new Cloud Router commands and parameters".

And also, as in other REST resource documents, I did not observe a section describing fields. The fact that NAT is called as a subcommand of the router command in the examples document gave (gcloud compute routers nats create) made me suspect that this resource is not a REST resource. When you keep looking at the reference document of NAT, it directs you to this link -> https://cloud.google.com/network-connectivity/docs/router When you follow the APIs from this link, you reach the API reference document of the Router REST resource again.

Then, when I review the Go SDK of the Google cloud provider, I see functions such as insert/create, delete, update for the Router resource (and of course for other REST resources), but we cannot see these functions for the NAT resource. So if these functions are not available for NAT, how is a NAT created? The first thing that came to my mind was that creating a NAT is the same as updating the NAT array on the Router. I did a test to see if this idea was correct.

First, I created a router called test-1 from the Google Cloud Platform UI. Then I stored the describe output of this Router resource from gcloud cli:

bgp:
  advertiseMode: DEFAULT
  asn: 0
  keepaliveInterval: 20
creationTimestamp: '2021-11-02T09:22:53.939-07:00'
description: ''
id: '***'
kind: compute#router
name: test-1
network: ***
region: ***
selfLink: ***/routers/test-1

Then again I created a NAT from the UI. Then, when I looked at the describe output of the test-1 Router object from the gcloud cli, I observed that this resource has been updated.

bgp:
  advertiseMode: DEFAULT
  asn: 0
  keepaliveInterval: 20
creationTimestamp: '2021-11-02T09:22:53.939-07:00'
description: ''
id: '***'
kind: compute#router
name: test-1
nats:
- enableEndpointIndependentMapping: false
  icmpIdleTimeoutSec: 30
  logConfig:
    enable: false
    filter: ALL
  minPortsPerVm: 64
  name: test-nat-1
  natIpAllocateOption: AUTO_ONLY
  sourceSubnetworkIpRangesToNat: ALL_SUBNETWORKS_ALL_IP_RANGES
  tcpEstablishedIdleTimeoutSec: 1200
  tcpTransitoryIdleTimeoutSec: 30
  udpIdleTimeoutSec: 30
network: ***
region: ***
selfLink: ***/routers/test-1

As it can be understood from here, creating a NAT means updating the NAT list inside the Router.

After this test, we (@ulucinar and me) looked at Terraform's provider-google repository and found that NAT was implemented as a separate resource. However, when we looked at the NAT's create function, we observed that the http operation is PATCH. (This is POST for Router or any other REST resource).

Create function of NAT (PATCH) -> https://github.com/hashicorp/terraform-provider-google/blob/0800026ac31dbeb8586364ff1b38e6434814d3ad/google/resource_compute_router_nat.go#L439

Read function of NAT (Reading router endpoint) -> https://github.com/hashicorp/terraform-provider-google/blob/0800026ac31dbeb8586364ff1b38e6434814d3ad/google/resource_compute_router_nat.go#L473

Create function of Router (POST) -> https://github.com/hashicorp/terraform-provider-google/blob/0800026ac31dbeb8586364ff1b38e6434814d3ad/google/resource_compute_router.go#L248

So I would like to pose the question: Although looks like NAT is not a REST resource, should NAT be a managed resource for Crossplane? Is it a reference point that Terraform defines NAT as a separate resource from Router?

My point of view is that, NAT should not be a separate managed resource and in order to create a NAT, the spec of a created Router must be updated. I'm curious about your thoughts. Thanks!

[muvaf]

@sergenyalcin thanks a lot for the detailed write-up! It seems like NAT is a resource belonging to Router in the API but it gcloud and terraform decided to treat it like a separate resource. Google DCL, which is what Terraform GCP provider is in the process of migrating to, seems closer to the API where the nats array exists as property of Router.

What we usually face is that the cloud API would expose a separate CRUD set as well as the array in an object, and in such cases we opt for making it a separate resource. If we're really in-between, I usually look up what Terraform community decided and go for that. In this case, API makes it a part of Router endpoint and nothing else. Since we're bound by the API, I agree that we should stick to keeping it as part of Router. Though it's interesting that gcloud CLI also treats it as separate resource; that's something new for me because the CLI commands are usually one-to-one match with the API.

[turkenh]

FWIW, just made another pass on the open PR adding Router and it looks like support for NAT is already implemented as part of Router there: https://github.com/crossplane/provider-gcp/pull/341/files#diff-3a20065320d8d1ed51b585c7687efc22871b491a427881b8c42dc7ab01d815e4R12

[laetho]

It's currently not possible to create a router and a nat from the Router resource alone. Atleast I have not figured out how to do it. The following occures, event + payload:

creation of Router resource has failed: googleapi: Error 400: Creating a router with multiple features is currently unsupported., badRequest

forProvider:
            bgp:
              advertiseMode: DEFAULT
            region: europe-north1
            networkRef: mynetwork
            nats:
              - name: cloud-nat
                minPortsPerVm: 128
                natIpAllocateOption: AUTO_ONLY
                enableEndpointIndependentMapping: false
                sourceSubnetworkIpRangesToNat: ALL_SUBNETWORKS_ALL_IP_RANGES

[wilhelmi]

On trying to setup a NAT router using v0.21.0 I am running into this issue:

apiVersion: compute.gcp.crossplane.io/v1alpha1
kind: Router
...
spec:
  forProvider:
    region: us-central1
    nats:
    - name: management
      natIpAllocateOption: "AUTO_ONLY"
      sourceSubnetworkIpRangesToNat: "LIST_OF_SUBNETWORKS"
      subnetworks:
      - name: ""
        sourceIpRangesToNat: ["PRIMARY_IP_RANGE"]

I am getting the following error:

cannot render composed resource from resource template at index 3: cannot use dry-run create to name composed resource: Router.compute.gcp.crossplane.io "test-environment-ldrsc" is invalid: spec.forProvider.nats.subnetworks.sourceIpRangesToNat: Unsupported value: []interface {}{"PRIMARY_IP_RANGE"}: supported values: "ALL_IP_RANGES", "LIST_OF_SECONDARY_IP_RANGES", "PRIMARY_IP_RANGE"

I read the Router CRD as an array, but trying it as a single string, I get this error:

cannot render composed resource from resource template at index 3: cannot use dry-run create to name composed resource: Router.compute.gcp.crossplane.io "test-environment-jr975" is invalid: spec.forProvider.nats.subnetworks.sourceIpRangesToNat: Invalid value: "string": spec.forProvider.nats.subnetworks.sourceIpRangesToNat in body must be of type array: "string"

Am I missing something here, or is this just not ready?