HTTPRoute automatic service intention config entry

[joshRooz]

Overview of the Issue

Applying an HTTPRoute automatically creates an accompanying service intention config entry for the destination service directly through the Consul API.

Attempting to manage the service intentions for the same destination service afterwards, using a ServiceIntentions custom resource, will silently fail as the Consul K8s controller responsible for reconciliation of ServiceIntentions does not merge an existing config entry.

Reproduction Steps

1. Create API-Gateway CRDs
2. helm install consul ...
3. Deploy API Gateway resource agw
4. Deploy public-api service
5. Create HTTPRoute for public-api
6. Deploy frontend service
7. Attempt to create ServiceIntentions for public-api to add frontend to sources array

Logs

Logs

config entry output after HTTPRoute creation:

#consul config read -kind=service-intentions -namespace=hashicups -name=public-api
{
    "Kind": "service-intentions",
    "Name": "public-api",
    "Partition": "default",
    "Namespace": "hashicups",
    "Sources": [
        {
            "Name": "agw",
            "Partition": "default",
            "Namespace": "api-gateway",
            "Action": "allow",
            "Precedence": 9,
            "Type": "consul",
            "Description": "Allow traffic from Consul API Gateway. Reconciled by controller at 2023-03-10T23:29:36Z."
        }
    ],
    "CreateIndex": 7252,
    "ModifyIndex": 7252
}

output from 'kubectl logs':

consul-connect-injector-7c8985d9fb-cpq6s sidecar-injector 2023-03-10T23:38:34.028Z	ERROR	controller.serviceintentions	Reconciler error	{"reconciler group": "consul.hashicorp.com", "reconciler kind": "ServiceIntentions", "name": "public-api", "namespace": "hashicups", "error": "config entry already exists in Consul"}
consul-connect-injector-7c8985d9fb-cpq6s sidecar-injector sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
consul-connect-injector-7c8985d9fb-cpq6s sidecar-injector 	/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266
consul-connect-injector-7c8985d9fb-cpq6s sidecar-injector sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
consul-connect-injector-7c8985d9fb-cpq6s sidecar-injector 	/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227
consul-connect-injector-7c8985d9fb-cpq6s sidecar-injector 2023-03-10T23:38:34.055Z	ERROR	controller.serviceintentions	sync failed	{"request": "hashicups/public-api", "error": "config entry already exists in Consul"}

Expected behavior

Capability to create an HTTPRoute while maintaining the ability to define [additional] service intentions for the same destination service through the ServiceIntentions CRD.

Environment details

• consul-api-gateway version: 0.5.1
• configuration used to deploy the gateway controller:

  # consul 1.0.4 helm chart
  apiGateway:
    enabled: true
    image: hashicorp/consul-api-gateway:0.5.1
    imageEnvoy: envoyproxy/envoy:v1.24.2
  

• Kubernetes version: v1.23.12
• Consul Server version: hashicorp/consul-enterprise:1.14.4-ent-ubi
• Consul-K8s version: hashicorp/consul-k8s-control-plane:1.0.4-ubi
• Cloud Provider: OpenShift 4.10.40
• Networking CNI plugin in use: OpenShift SDN with Multus + Consul CNI

[codex70]

I can confirm I'm seeing the same issue, the ServiceIntentions appears to deploy correctly, however when I describe the ServiceIntentions I see the following:

Status:
  Conditions:
    Last Transition Time:  2023-03-17T16:39:57Z
    Message:               config entry already exists in Consul
    Reason:                ExternallyManagedConfigError
    Status:                False
    Type:                  Synced

[nathancoleman]

Hi there @joshRooz @codex70 👋

Consul API Gateway has evolved a lot this year and now comes baked into the consul-k8s Helm chart without the need for this separate project/controller. As a result, the project in this repo has been deprecated (see this release note).

The specific issue described here has been addressed in v1.2.0+ of consul-k8s, and the API gateway solution there does not create intentions, instead leaving that to the user to do however they wish and avoiding the need to merge the configuration entries altogether (see release note).

We recommend upgrading to v1.2.0+ of the Consul Helm chart and running Consul v1.16.0+ with the native Consul API Gateway solution baked in to resolve this issue. You can find instructions for completing that upgrade here.