kubernetes-sigs
No endpoints could be generated from HTTPRoute for no reason
time="2024-05-07T03:58:32Z" level=debug msg="No endpoints could be generated from HTTPRoute foo/bar"
And that's it. There's no other logs that can shed any light on why it cannot be generated. Looking at this atrocity I see a lot of places where it can go wrong, but zero debug logging. Nice!
Namespace discrepancy between Gateway and HttpRoute. eh
Why not print a log line on that?
We also stumbled about this. But we would really like to be able define gateways and route in different namespaces.
@pkit Do you know if Is this not possible (rbac or whatever) or just not implemented as a feature?
Gateways can have many different Listeners and Routes aren't required to match all of them. The logs would be really noisy if every non-match was logged when this is a commonly expected case. However, it is not expected for a Gateway to have accepted a Route AND for no Listener match to be found. This case is logged in the code you linked.
https://github.com/kubernetes-sigs/external-dns/blob/fa17f9d06cbfe3315430529151324fae5a421e43/source/gateway.go#L360
It also logs several other potential issues...
- If the parent is an unknown/unsupported type (i.e. not a Gateway): https://github.com/kubernetes-sigs/external-dns/blob/fa17f9d06cbfe3315430529151324fae5a421e43/source/gateway.go#L298
- If the Gateway couldn't be found: https://github.com/kubernetes-sigs/external-dns/blob/fa17f9d06cbfe3315430529151324fae5a421e43/source/gateway.go#L305
- If the Gateway is attached to the Route but hasn't accepted it: https://github.com/kubernetes-sigs/external-dns/blob/fa17f9d06cbfe3315430529151324fae5a421e43/source/gateway.go#L310
Gateways, Routes, and Services can cross namespace boundaries, but this must be explicitly allowed. It doesn't happen by default. In your case, @pkit, this is why the Gateway never attached to the Route. Please see the Gateway API's security model for more details (cc @jakoberpf).
I think external-dns should do its best to convert the state of your cluster resources to DNS entries. I don't think it should try to determine the user's true intentions and diagnose misconfigurations of resource specifications. It just takes what it's given.
That said, since it's already logging that no endpoints were found it could also specify that no Gateways were attached.
@abursavich Thanks for elaborating that. Indeed providing the right namespace allow selectors on the gateway fixed my issue.
Faced completely the same issue. For instance, I want to terminate TLS on GW side, so there is certificate attached to listener section.
listeners:
- allowedRoutes:
namespaces:
from: All
name: web-secure
port: 8443
hostname: example.com
protocol: HTTPS
tls:
certificateRefs:
- kind: Secret
group: ""
name: example-secret
mode: Terminate
So on GW side it is HTTPS protocol, while with HTTPRoute there is no protocol (per se) that you can specify.
Now why I think DNS cannot be provisioned.
It is b/c of this function.
There it says if protocols do not match between GW & HTTPRoute that route will be just skipped.
// Confirm that the Listener and Route protocols match.
if !gwProtocolMatches(rt.Protocol(), lis.Protocol) {
continue
}
Regarding HTTPRoute the only protocol is the one defined by kubernetes service and when web based it is TCP.
Can you advise which protocol is taken into consideration on HTTPRoute side, or perhaps it can be good enough if that condition would be omitted? Thanks.
@zargor, HTTPRoute matches both HTTP and HTTPS protocols on the Listener.
The first thing to look for is if the Route has been accepted by the Gateway. You can check the Route status for that. If the Gateway hasn't accepted the Route the Gateway won't serve the Route, so ExternalDNS won't match them together either. If the Route has been accepted by the Gateway, then there might be a problem with ExternalDNS. In either case, I'd be happy to help you debug your issue if you provide the full Gateway and HTTPRoute resources, including status. Feel free to obscure the IPs and hostnames if desired, but not so much that it changes the hostname matching semantics.
Sure @abursavich , thanks, here are described gateway and httroute respectfully:
Gateway:
Name: traefik-g2
Namespace: traefik
Labels: app.kubernetes.io/instance=traefik-traefik
app.kubernetes.io/name=traefik
Annotations: <none>
API Version: gateway.networking.k8s.io/v1
Kind: Gateway
Metadata:
Creation Timestamp: 2024-08-30T14:27:54Z
Generation: 1
Resource Version: 1361676557
UID: cecaf3f9-ecaa-495e-805d-0856580077f7
Spec:
Gateway Class Name: traefik
Listeners:
Allowed Routes:
Namespaces:
From: All
Name: web
Port: 8000
Protocol: HTTP
Allowed Routes:
Namespaces:
From: All
Name: web-mtls
Port: 8443
Protocol: HTTPS
Tls:
Certificate Refs:
Group:
Kind: Secret
Name: server-secret
Mode: Terminate
Status:
Conditions:
Last Transition Time: 2024-08-30T14:27:54Z
Message: Gateway successfully scheduled
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Last Transition Time: 2024-08-30T14:27:54Z
Message: Gateway successfully scheduled
Observed Generation: 1
Reason: Programmed
Status: True
Type: Programmed
Listeners:
Attached Routes: 0
Conditions:
Last Transition Time: 2024-08-30T14:27:54Z
Message: No error found
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Last Transition Time: 2024-08-30T14:27:54Z
Message: No error found
Observed Generation: 1
Reason: ResolvedRefs
Status: True
Type: ResolvedRefs
Last Transition Time: 2024-08-30T14:27:54Z
Message: No error found
Observed Generation: 1
Reason: Programmed
Status: True
Type: Programmed
Name: web
Supported Kinds:
Group: gateway.networking.k8s.io
Kind: HTTPRoute
Attached Routes: 1
Conditions:
Last Transition Time: 2024-08-30T14:27:54Z
Message: No error found
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Last Transition Time: 2024-08-30T14:27:54Z
Message: No error found
Observed Generation: 1
Reason: ResolvedRefs
Status: True
Type: ResolvedRefs
Last Transition Time: 2024-08-30T14:27:54Z
Message: No error found
Observed Generation: 1
Reason: Programmed
Status: True
Type: Programmed
Name: web-mtls
Supported Kinds:
Group: gateway.networking.k8s.io
Kind: HTTPRoute
Events: <none>
HTTPRoute:
Name: route-ws
Namespace: some-namespace
Labels: <none>
Annotations: external-dns.alpha.kubernetes.io/hostname: example.com
API Version: gateway.networking.k8s.io/v1
Kind: HTTPRoute
Metadata:
Creation Timestamp: 2024-08-29T12:22:29Z
Generation: 1
Resource Version: 1361676554
UID: b5535d72-5b5e-400f-80f3-9654f21f92a3
Spec:
Hostnames:
example.com
Parent Refs:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: traefik-g2
Namespace: traefik
Port: 8443
Section Name: web-mtls
Rules:
Backend Refs:
Group:
Kind: Service
Name: ws-deployment
Port: 8443
Weight: 1
Matches:
Path:
Type: PathPrefix
Value: /
Status:
Parents:
Conditions:
Last Transition Time: 2024-08-30T14:27:54Z
Message:
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Last Transition Time: 2024-08-30T14:27:54Z
Message:
Observed Generation: 1
Reason: ResolvedRefs
Status: True
Type: ResolvedRefs
Controller Name: traefik.io/gateway-controller
Parent Ref:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: traefik-g2
Namespace: traefik
Port: 8443
Section Name: web-mtls
Events: <none>
@zargor, the only thing that's jumping out at me is that the Gateway doesn't have any IP addresses in its status. The Gateway and the HTTPRoute are attached and ExternalDNS should agree, but it doesn't have any addresses to use for DNS records.
https://github.com/kubernetes-sigs/external-dns/blob/master/docs/sources/gateway.md#targets
@zargor, the only thing that's jumping out at me is that the Gateway doesn't have any IP addresses in its status. The Gateway and the HTTPRoute are attached and ExternalDNS should agree, but it doesn't have any addresses to use for DNS records.
https://github.com/kubernetes-sigs/external-dns/blob/master/docs/sources/gateway.md#targets
In this case, it is about external NLB which comes with its own generic/default dns record. But yes, I suppose there can be an issue, since gateway Address is not populated. On the other hand ,my sense is that external dns should try to create CNAME based on the k8s service, which is responsible for NLB.