serving Knative routing of cluster local host names does not match k8s dns behavior

In what area(s)?

/area API /area networking

What version of Knative?

1.6 with Istio

Expected Behavior

Assume that the foobar service is deployed to the baz namespace, and we are trying to call foobar from another pod in the baz namespace. Calls of the form curl foobar should work with knative, the same as with a normal k8s Service, which will automatically route to foobar.baz.

Actual Behavior

A 404 error is returned, as the service cannot be routed. Calls with the namespace included like curl foobar.baz will work as expected. This appears to be due to the way the Istio Gateway, VirtualService, or KIngresses are configured.

This should at least be documented since it breaks the notion of Knative hiding Kubernetes abstraction details. It's not clear to me if this is easy to fix, since the gateway would have to know or discover the namespace of the client. The workaround is to change all clients to use the hostname with a namespace.

For users that are migrating existing K8S services to Knative, this can be a surprise since foobar.baz and foobar.baz.svc.cluster.local work normally.

Steps to Reproduce the Problem

Deploy an image using a K8s Deployment and Service. Shell into the Pod and curl the service without the namespace. It should succeed.
Delete the K8s Deployment and Service, and re-deploy the image as a Knative service, using the same name. Curl it without the namespace, and it should return 404.

Nov 28 '22 03:11 xtaje

I deploy with istio and did the following test:

root@cesign [08:12:33 PM] [+29.0°C] [~/git/group-service-acceleration/config] [main *]
-> # k get vs
NAME                           GATEWAYS                                    HOSTS                                           AGE
group-service-c-ingress-test   ["knative-serving/knative-local-gateway"]   ["group-service-c","group-service-c.default"]   24h
root@cesign [08:12:44 PM] [+29.0°C] [~/git/group-service-acceleration/config] [main *]
-> # 
root@cesign [08:14:09 PM] [+28.0°C] [~/git/group-service-acceleration/config] [main *]
-> # k exec mycurlpod -- curl group-service-c.default    
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   615  100   615    0     0   4592      0 --:--:-- --:--:-- --:--:--  4624
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://[nginx.org](http://nginx.org/)/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://[nginx.com](http://nginx.com/)/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@cesign [08:14:15 PM] [+29.0°C] [~/git/group-service-acceleration/config] [main *]
-> # k exec mycurlpod -- curl group-service-c         
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

It looks like the istio could not distinguish group-service-c host header.

But from official website: https://istio.io/latest/docs/reference/config/networking/virtual-service/

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews-route
  namespace: foo
spec:
  hosts:
  - reviews # interpreted as reviews.foo.svc.cluster.local
  http:
....

It could route even it's group-service-c.

Dec 06 '22 12:12 jwcesign

This issue is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen the issue with /reopen. Mark the issue as fresh by adding the comment /remove-lifecycle stale.

Mar 07 '23 01:03 github-actions[bot]

/lifecycle frozen

Mar 29 '23 16:03 dprotaso

@dprotaso after reading this issue and the related one I don't think there is anything we can do here with central ingress controllers/gateways/...? I would propose closing this issue and accepting this limitation.

Dec 05 '25 15:12 linkvt