Move annotations from HttpRoute to the Gateway resource

[Andrei-Predoiu]

trafficstars

My team has ran an experimental migration of our k8s platform from ingress to gateway api. external-dns is one of the key areas we had issues in.

What would you like to be added:

Currently external-dns is implemented to use the HttpRoute resource to provision DNS. I suggest moving the annotation support to the Gateway resource instead.

Why is this needed:

1. The Gateway API feature is designed in a way that breaks responsibility down to different layers. The HttpRoute resource is meant to be used by app devs, while the DNS of the gateway falls more in the ClusterOperator role.
2. The whole idea of a gateway is a single entry point(host) which multiple backends can share. I can see the utility of of allowing a mapping between hostnames and HttpRoutes but it's not what the gateway was designed to do. I would suggest that individuals with this use case either use ingress, configure multiple gateway instances, or use route filters/rules.
3. Inside k8s, the Gateway or GatewayClass resources are the ones that have the actual external IP of the gateway, definitely not the HttpRoute. In our experiment, the IP detected by external-dns was wrong. (I forgot to write down exactly where the real IP was, sorry)
4. Cert Manager is a project that is almost always deployed in tandem with external-dns. They annotate the Gateway resource, thus these tools don't really work the same way any more in the Gateway API realm

[dmuiX]

Actually its already possible to use these annotations in a gateway. Just try it you will be surprised. Not sure when they added that. And its also not documented anywhere, at least I just have found what you wrote above. And whats also not documented it that you need to add the httproute and the tlsroute as sources either in a args/cmd field or in a helm chart.