aws-app-mesh-controller-for-k8s icon indicating copy to clipboard operation
aws-app-mesh-controller-for-k8s copied to clipboard

Cannot update VirtualRouter listener protocol with defined routes

Open dhild opened this issue 3 months ago • 0 comments

Describe the bug If I replace the listener protocol and a matching route's protocol in a VirtualRouter at the same time, the change is accepted by the API server but fails to reconcile.

Steps to reproduce

  1. Create a VirtualRouter with a listener and a matching route. Wait for the status to show that the VirtualRouter is successfully reconciled.
kind: VirtualRouter
apiVersion: appmesh.k8s.aws/v1beta2
metadata:
  name: example-router
spec:
  listeners:
  - portMapping:
      protocol: http
      port: 8080
  routes:
  - name: web
    httpRoute:
      match:
        prefix: /
        port: 8080
      action:
        weightedTargets:
        - virtualNodeRef:
            name: example-node
          port: 8080
          weight: 1
  1. Change the listener protocol, and update the protocol used for the route to match. This change will be accepted by the API server.
kind: VirtualRouter
apiVersion: appmesh.k8s.aws/v1beta2
metadata:
  name: example-router
spec:
  listeners:
  - portMapping:
      protocol: http2
      port: 8080
  routes:
  - name: web
    http2Route:
      match:
        prefix: /
        port: 8080
      action:
        weightedTargets:
        - virtualNodeRef:
            name: example-node
          port: 8080
          weight: 1
  1. Warning events can be observed on the VirtualRouter, and the status never transitions to observe the latest generation.
  Warning  ReconcileError  5s   VirtualRouter  BadRequestException: 1 Virtual Router listener(s) cannot be removed because they are matched by existing Routes. Listing up to 5 PortMappings: [(Port: 8080, Protocol: HTTP)]

Workaround Manually sequence the updates, by first removing the route before changing the listener protocol. Once this has reconciled, apply the desired final spec.

kind: VirtualRouter
apiVersion: appmesh.k8s.aws/v1beta2
metadata:
  name: example-router
spec:
  listeners:
  - portMapping:
      protocol: http
      port: 8080
  routes: []

Expected outcome The controller should perform the API calls in the same sequence as the workaround, leaving the AWS resources in the desired state.

Environment

  • App Mesh controller version: v1.12.1
  • Envoy version: v1.25.4.0-prod
  • Are you using any integrations? X-ray, Jaeger etc. If so versions? N/A
  • Kubernetes version v1.27.10
  • Using EKS (yes/no), if so version? v1.27.10-eks-508b6b3

Additional Context: The same error can also be reproduced by removing both the listener and the route at the same time.

dhild avatar Mar 21 '24 23:03 dhild