gateway
gateway copied to clipboard
envoyproxy
Policy with targetSelector targeting multiple gateways does not show both ancestors
Description: When targeting resources across multiple gateways with targetSelectors, the status of the policy will only contain one gateway, and it will continously rotate between the targeted gateways
Repro steps: Create two gateways, attach two httproutes to the gateways(one per), and use targetSelector to target one policy(We see it with BackendTrafficPolicy and SecurityPolicy) to both httproutes
GW-1 <- HTTPRoute-1
\
BackendTrafficPolicy
/
GW-2 <- HTTPRoute-2
Environment: Seeing it both on 1.3.0 and 1.4.0
We did previously see a similar issue attaching a HTTPRoute to multiple gateways which was solved in https://github.com/envoyproxy/gateway/issues/4264
Hi @BadLiveware I tested with a similar setup and everything looks correct on my end. Could you share the specific configuration(with status) you’re having trouble with?
backendTrafficPolicies:
- apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
creationTimestamp: null
name: foo
namespace: default
spec:
targetSelectors:
- kind: HTTPRoute
matchLabels:
foo: bar
timeout:
http:
connectionIdleTimeout: 21s
tcp:
connectTimeout: 20s
status:
ancestors:
- ancestorRef:
group: gateway.networking.k8s.io
kind: Gateway
name: gateway-1
namespace: default
conditions:
- lastTransitionTime: null
message: Policy has been accepted.
reason: Accepted
status: "True"
type: Accepted
controllerName: gateway.envoyproxy.io/gatewayclass-controller
- ancestorRef:
group: gateway.networking.k8s.io
kind: Gateway
name: gateway-2
namespace: default
conditions:
- lastTransitionTime: null
message: Policy has been accepted.
reason: Accepted
status: "True"
type: Accepted
controllerName: gateway.envoyproxy.io/gatewayclass-controller
gateways:
- apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
creationTimestamp: null
name: gateway-1
namespace: default
spec:
gatewayClassName: envoy-gateway-class
listeners:
- name: http
port: 80
protocol: HTTP
status:
listeners:
- attachedRoutes: 1
conditions:
- lastTransitionTime: null
message: Sending translated listener configuration to the data plane
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: null
message: Listener has been successfully translated
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: null
message: Listener references have been resolved
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
creationTimestamp: null
name: gateway-2
namespace: default
spec:
gatewayClassName: envoy-gateway-class
listeners:
- name: http
port: 80
protocol: HTTP
status:
listeners:
- attachedRoutes: 1
conditions:
- lastTransitionTime: null
message: Sending translated listener configuration to the data plane
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: null
message: Listener has been successfully translated
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: null
message: Listener references have been resolved
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
httpRoutes:
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
creationTimestamp: null
labels:
foo: bar
name: gateway-1-httproute-1
namespace: default
spec:
parentRefs:
- name: gateway-1
namespace: default
rules:
- backendRefs:
- name: service-1
port: 8080
status:
parents:
- conditions:
- lastTransitionTime: null
message: Route is accepted
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: null
message: Resolved all the Object references for the Route
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parentRef:
name: gateway-1
namespace: default
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
creationTimestamp: null
labels:
foo: bar
name: gateway-2-httproute-1
namespace: default
spec:
parentRefs:
- name: gateway-2
namespace: default
rules:
- backendRefs:
- name: service-1
port: 8080
status:
parents:
- conditions:
- lastTransitionTime: null
message: Route is accepted
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: null
message: Resolved all the Object references for the Route
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parentRef:
name: gateway-2
namespace: default
Absolutely, this is an excerpt of relevant resources. This is running on 1.4.0
gateways:
- apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
annotations:
argocd.argoproj.io/tracking-id: envoy-gateway-dev-selling:gateway.networking.k8s.io/Gateway:envoy-gateway-system/external-iap
cert-manager.io/cluster-issuer: google
external-dns.alpha.kubernetes.io/target: external-iap.dev-selling.example.dev
creationTimestamp: "2025-03-17T08:27:56Z"
generation: 1
name: external-iap
namespace: envoy-gateway-system
resourceVersion: "196475925"
uid: 2d7a96b6-56c9-40de-9d1f-7e4633b7cc05
spec:
gatewayClassName: external-iap
listeners:
- allowedRoutes:
namespaces:
from: All
name: http
port: 80
protocol: HTTP
status:
addresses:
- type: IPAddress
value: 10.194.63.168
conditions:
- lastTransitionTime: "2025-05-20T15:59:43Z"
message: The Gateway has been scheduled by Envoy Gateway
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T15:59:43Z"
message: Address assigned to the Gateway, 2/2 envoy replicas available
observedGeneration: 1
reason: Programmed
status: "True"
type: Programmed
listeners:
- attachedRoutes: 126
conditions:
- lastTransitionTime: "2025-05-20T15:59:42Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 1
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T15:59:42Z"
message: Listener has been successfully translated
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T15:59:42Z"
message: Listener references have been resolved
observedGeneration: 1
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
annotations:
argocd.argoproj.io/tracking-id: envoy-gateway-dev-selling:gateway.networking.k8s.io/Gateway:envoy-gateway-system/internal
cert-manager.io/cluster-issuer: google
creationTimestamp: "2025-03-17T08:28:01Z"
generation: 2
name: internal
namespace: envoy-gateway-system
resourceVersion: "196480347"
uid: 30862929-eae8-4eee-b889-e4e64dd8ce67
spec:
gatewayClassName: internal
listeners:
- allowedRoutes:
namespaces:
from: All
hostname: '*.dev-selling.example.service'
name: http-wildcard-dev-selling-example-service
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: All
hostname: '*.dev-selling.i.example.service'
name: http-wildcard-dev-selling-i-example-service
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: Same
hostname: '*.dev-selling.example.dev'
name: http-wildcard-dev-selling-example-dev
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: All
hostname: '*.dev-selling.example.dev'
name: https-wildcard-dev-selling-example-dev
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: internal-wildcard-dev-selling-example-dev
mode: Terminate
- allowedRoutes:
namespaces:
from: Same
hostname: dev-selling.example.dev
name: http-dev-selling-example-dev
port: 80
protocol: HTTP
- allowedRoutes:
namespaces:
from: All
hostname: dev-selling.example.dev
name: https-dev-selling-example-dev
port: 443
protocol: HTTPS
tls:
certificateRefs:
- group: ""
kind: Secret
name: internal-dev-selling-example-dev
mode: Terminate
status:
addresses:
- type: IPAddress
value: 10.98.189.206
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: The Gateway has been scheduled by Envoy Gateway
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Address assigned to the Gateway, 2/2 envoy replicas available
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
listeners:
- attachedRoutes: 261
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener has been successfully translated
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener references have been resolved
observedGeneration: 2
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http-wildcard-dev-selling-example-service
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 261
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener has been successfully translated
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener references have been resolved
observedGeneration: 2
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http-wildcard-dev-selling-i-example-service
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener has been successfully translated
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener references have been resolved
observedGeneration: 2
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http-wildcard-dev-selling-example-dev
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 261
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener has been successfully translated
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener references have been resolved
observedGeneration: 2
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: The certificate san *.dev-selling.example.dev overlaps with the certificate
san dev-selling.example.dev in listener https-dev-selling-example-dev. ALPN
is set to HTTP/1.1 to prevent HTTP/2 connection coalescing
observedGeneration: 2
reason: OverlappingCertificates
status: "True"
type: OverlappingTLSConfig
name: https-wildcard-dev-selling-example-dev
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 1
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener has been successfully translated
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener references have been resolved
observedGeneration: 2
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
name: http-dev-selling-example-dev
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
- attachedRoutes: 261
conditions:
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Sending translated listener configuration to the data plane
observedGeneration: 2
reason: Programmed
status: "True"
type: Programmed
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener has been successfully translated
observedGeneration: 2
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: Listener references have been resolved
observedGeneration: 2
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
- lastTransitionTime: "2025-05-20T16:01:36Z"
message: The certificate san dev-selling.example.dev overlaps with the certificate
san *.dev-selling.example.dev in listener https-wildcard-dev-selling-example-dev.
ALPN is set to HTTP/1.1 to prevent HTTP/2 connection coalescing
observedGeneration: 2
reason: OverlappingCertificates
status: "True"
type: OverlappingTLSConfig
name: https-dev-selling-example-dev
supportedKinds:
- group: gateway.networking.k8s.io
kind: HTTPRoute
- group: gateway.networking.k8s.io
kind: GRPCRoute
backendtrafficpolicy:
- apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
annotations:
argocd.argoproj.io/tracking-id: www-proxy-replacement-dev-selling:gateway.envoyproxy.io/BackendTrafficPolicy:development/www-proxy-replacement-retry
creationTimestamp: "2025-05-21T07:43:10Z"
generation: 1
name: www-proxy-replacement-retry
namespace: development
resourceVersion: "198021737"
uid: df2a6692-b348-4f11-af54-5cf7f46d24eb
spec:
healthCheck:
passive:
baseEjectionTime: 15s
consecutive5XxErrors: 5
consecutiveGatewayErrors: 5
consecutiveLocalOriginFailures: 5
interval: 3s
maxEjectionPercent: 10
splitExternalLocalOriginErrors: false
retry:
numRetries: 2
perRetry:
backOff:
baseInterval: 100ms
maxInterval: 10s
retryOn:
triggers:
- connect-failure
- gateway-error
- refused-stream
- reset
targetSelectors:
- group: gateway.networking.k8s.io
kind: HTTPRoute
matchLabels:
envoy-retry: www-proxy-replacement
timeout:
http:
requestTimeout: 30s
tcp:
connectTimeout: 2s
status:
ancestors:
- ancestorRef:
group: gateway.networking.k8s.io
kind: Gateway
name: external-iap
namespace: envoy-gateway-system
conditions:
- lastTransitionTime: "2025-05-21T08:13:27Z"
message: Policy has been accepted.
reason: Accepted
status: "True"
type: Accepted
controllerName: gateway.envoyproxy.io/gatewayclass-controller
httproutes:
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
annotations:
argocd.argoproj.io/tracking-id: www-proxy-replacement-dev-selling:gateway.networking.k8s.io/HTTPRoute:development/www-proxy-replacement-info-web-internal
external-dns.alpha.kubernetes.io/selector: new-xtdns
creationTimestamp: "2025-05-20T15:39:47Z"
generation: 1
labels:
envoy-retry: www-proxy-replacement
name: www-proxy-replacement-info-web-internal
namespace: development
resourceVersion: "196415242"
uid: 12b3b71c-8c1d-4637-a2eb-c1dafa5edaa1
spec:
hostnames:
- www.dev-selling.example.dev
- www-proxy.dev-selling.example.service
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: internal
namespace: envoy-gateway-system
rules:
- backendRefs:
- group: ""
kind: Service
name: info-web
port: 80
weight: 1
matches:
- path:
type: Exact
value: /support
- path:
type: PathPrefix
value: /support/
timeouts:
backendRequest: 60s
request: 60s
status:
parents:
- conditions:
- lastTransitionTime: "2025-05-20T15:39:48Z"
message: Route is accepted
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T15:39:48Z"
message: Resolved all the Object references for the Route
observedGeneration: 1
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parentRef:
group: gateway.networking.k8s.io
kind: Gateway
name: internal
namespace: envoy-gateway-system
- apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
annotations:
argocd.argoproj.io/tracking-id: www-proxy-replacement-dev-selling:gateway.networking.k8s.io/HTTPRoute:development/www-proxy-replacement-info-web-external-iap
external-dns.alpha.kubernetes.io/selector: new-xtdns
creationTimestamp: "2025-05-20T15:39:47Z"
generation: 1
labels:
envoy-retry: www-proxy-replacement
name: www-proxy-replacement-info-web-external-iap
namespace: development
resourceVersion: "196415414"
uid: 569bad7a-15db-4058-aee8-43cf9f72c1dd
spec:
hostnames:
- www.dev-selling.example.dev
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: external-iap
namespace: envoy-gateway-system
rules:
- backendRefs:
- group: ""
kind: Service
name: info-web
port: 80
weight: 1
matches:
- path:
type: Exact
value: /support
- path:
type: PathPrefix
value: /support/
timeouts:
backendRequest: 60s
request: 60s
status:
parents:
- conditions:
- lastTransitionTime: "2025-05-20T15:39:48Z"
message: Route is accepted
observedGeneration: 1
reason: Accepted
status: "True"
type: Accepted
- lastTransitionTime: "2025-05-20T15:39:48Z"
message: Resolved all the Object references for the Route
observedGeneration: 1
reason: ResolvedRefs
status: "True"
type: ResolvedRefs
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parentRef:
group: gateway.networking.k8s.io
kind: Gateway
name: external-iap
namespace: envoy-gateway-system
and here is the BackendTrafficPolicy switching its status to the other gateway
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
annotations:
argocd.argoproj.io/tracking-id: www-proxy-replacement-dev-selling:gateway.envoyproxy.io/BackendTrafficPolicy:development/www-proxy-replacement-retry
creationTimestamp: "2025-05-21T07:43:10Z"
generation: 1
name: www-proxy-replacement-retry
namespace: development
resourceVersion: "198043905"
uid: df2a6692-b348-4f11-af54-5cf7f46d24eb
spec:
healthCheck:
passive:
baseEjectionTime: 15s
consecutive5XxErrors: 5
consecutiveGatewayErrors: 5
consecutiveLocalOriginFailures: 5
interval: 3s
maxEjectionPercent: 10
splitExternalLocalOriginErrors: false
retry:
numRetries: 2
perRetry:
backOff:
baseInterval: 100ms
maxInterval: 10s
retryOn:
triggers:
- connect-failure
- gateway-error
- refused-stream
- reset
targetSelectors:
- group: gateway.networking.k8s.io
kind: HTTPRoute
matchLabels:
envoy-retry: www-proxy-replacement
timeout:
http:
requestTimeout: 30s
tcp:
connectTimeout: 2s
status:
ancestors:
- ancestorRef:
group: gateway.networking.k8s.io
kind: Gateway
name: internal
namespace: envoy-gateway-system
conditions:
- lastTransitionTime: "2025-05-21T08:26:47Z"
message: Policy has been accepted.
reason: Accepted
status: "True"
type: Accepted
controllerName: gateway.envoyproxy.io/gatewayclass-controller
This is a multi-writing issue from the status updator. An xPolicy can target resources tracing back to different GatewayClasses, and each GatewayClass has its own translator, which updates the status individually and overrides each other.
This issue has been automatically marked as stale because it has not had activity in the last 30 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been automatically marked as stale because it has not had activity in the last 30 days.