HTTP request transform type in Composition

[muvaf]

What problem are you facing?

There are bunch of patch and transformer types today but they don't cover (or aim to cover) all cases. So far, we've investigated CustomComposition path for people who want to achieve more customization over the composed resources. While that's a viable approach, it requires users to depart from Composition semantics completely. For a user who just wants to get an IP from another system, writing the full CustomComposition web server can be daunting. For example, the CustomComposition server will likely be like a black box and you need to find its source code to see what's going to happen when you create a claim as opposed to running kubectl. Such costs are higher than the benefits if what you want to customize is a very simple thing.

How could Crossplane help solve your problem?

We could have a transform type that sends a request to an HTTP endpoint and return the value to be used in the composed resource.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: compositepostgresqlinstances.aws.database.example.org
  labels:
    provider: aws
    guide: quickstart
    vpc: default
spec:
  writeConnectionSecretsToNamespace: crossplane-system
  compositeTypeRef:
    apiVersion: database.example.org/v1alpha1
    kind: CompositePostgreSQLInstance
  resources:
    - base:
        apiVersion: database.aws.crossplane.io/v1beta1
        kind: RDSInstance
        spec:
          forProvider:
            region: us-east-1
            dbInstanceClass: db.t2.small
            masterUsername: masteruser
            engine: postgres
            engineVersion: "9.6"
            skipFinalSnapshotBeforeDeletion: true
            publiclyAccessible: true
          writeConnectionSecretToRef:
            namespace: crossplane-system
      patches:
        - fromFieldPath: "spec.tier"
          toFieldPath: "spec.forProvider.vpcSecurityGroupID"
          # Crossplane will send a POST request to that URL with the value of `spec.tier` and the body will contain a string
          # that will be used in `spec.forProvider.vpcSecurityGroupID`
          transforms:
          - type: http
            http:
              url: "my.little.lambda.com:8080/vpc-resolver"
              # fields for authorization context/token/TLS secret etc. could be added here alongside url

This will allow people to add any kind of custom logic that can't be achieved with composition primitives to their HTTP server and that'll be the only thing they need to maintain; not the whole composition members like CustomComposition. Another upside of this approach compared to CustomComposition is that all composition features will apply smoothly, like CompositionRevision stuff.

[jbhennin]

As resources like security group ids are needed to create an RDS instance, etc. this feature is greatly needed IMO. Big thumbs up if something like this could get added to the roadmap.

[negz]

I feel like another good place for web hooks to slot in would be something like provider-webhook that could be used with custom composition to basically call an arbitrary REST endpoint. 🤔 Slightly different use case to what is mentioned here.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 7 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[github-actions[bot]]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 7 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.