DNS between k8's and mesos.

[timothysc]

The gulf between name resolution of services which may inter-operate needs to be bridged.

http://mesosphere.com/2015/01/21/mesos-dns-service-discovery/ vs. https://github.com/GoogleCloudPlatform/kubernetes/tree/master/cluster/addons/dns

[jdef]

xref https://github.com/mesosphere/mesos-dns/issues/62

[jdef]

#256 added discovery-info generation for tasks: experimental support discovery of pods (nothing yet for services)

[akamalov]

@jdef is pod lookup supported using Mesos-DNS ? I can for sure ping kubernetes.mesos, but can't seem to discover any pods (sample nginx running, can't discover).

[jdef]

experimental support was added via the scheduler flag: mesos-generate-task-discovery. it defaults to false. once enabled you can fine-tune its behavior by setting labels and/or attributes.

https://github.com/kubernetes/kubernetes/blob/master/contrib/mesos/pkg/scheduler/podtask/pod_task.go#L138

We haven't received much feedback on this feature yet, so if it's something you find useful please post a follow-up.

On Thu, Mar 31, 2016 at 12:45 PM, akamalov [email protected] wrote:

@jdef https://github.com/jdef is pod lookup supported using Mesos-DNS ? I can for sure ping kubernetes.mesos, but can't seem to discover any pods (sample nginx running, can't discover).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/mesosphere/kubernetes-mesos/issues/125#issuecomment-204015467

[jdef]

probably an easier read: https://github.com/kubernetes/kubernetes/blob/master/contrib/mesos/docs/discovery.md

On Thu, Mar 31, 2016 at 3:59 PM, James DeFelice [email protected] wrote:

experimental support was added via the scheduler flag: mesos-generate-task-discovery. it defaults to false. once enabled you can fine-tune its behavior by setting labels and/or attributes.

https://github.com/kubernetes/kubernetes/blob/master/contrib/mesos/pkg/scheduler/podtask/pod_task.go#L138

We haven't received much feedback on this feature yet, so if it's something you find useful please post a follow-up.

On Thu, Mar 31, 2016 at 12:45 PM, akamalov [email protected] wrote:

@jdef https://github.com/jdef is pod lookup supported using Mesos-DNS ? I can for sure ping kubernetes.mesos, but can't seem to discover any pods (sample nginx running, can't discover).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/mesosphere/kubernetes-mesos/issues/125#issuecomment-204015467

[akamalov]

@jdef, thanks so much!

[akamalov]

@jdef - Hey James. Modified and re-launched km-scheduler with --mesos-generate-task-discovery=true.

Deployed a sample application:

apiVersion: v1
kind: Service
metadata:
  name: nginxsvc
  labels:
    app: nginx-http
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx
---
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-rc
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
      annotations:
        k8s.mesosphere.io/discovery-name: nginx-rc
    spec:
      containers:
      - name: nginx-http
        image: nginx
        ports:
        - containerPort: 80

Launched application:

[root@node236 k8s]# kubectl create -f nginx-https2.yaml 
You have exposed your service on an external port on all nodes in your
cluster.  If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:30005) to serve traffic.

See http://releases.k8s.io/HEAD/docs/user-guide/services-firewalls.md for more details.
service "nginxsvc" created
replicationcontroller "nginx-rc" created

Check the status of pods:

[root@node236 k8s]# kubectl get pods -o wide
NAME             READY     STATUS    RESTARTS   AGE       NODE
nginx-rc-5uqd3   1/1       Running   0          8s        node261.local.net
[root@node236 k8s]# 

Display services:

[root@node236 k8s]# kubectl get services -o wide
NAME             CLUSTER-IP      EXTERNAL-IP   PORT(S)     AGE       SELECTOR
k8sm-scheduler   172.16.10.161   <none>        10251/TCP   1d        <none>
kubernetes       172.16.10.1     <none>        443/TCP     1d        <none>
nginxsvc         172.16.10.80    nodes         80/TCP      33s       app=nginx
[root@node236 k8s]# 

Try to nslookup the pod - format: ${name}.${pod-namespace}.pod:

[root@node236 k8s]# nslookup nginx-rc-5uqd3.default.pod
Server:         127.0.0.1
Address:        127.0.0.1#53

** server can't find nginx-rc-5uqd3.default.pod: NXDOMAIN

[root@node236 k8s]# 

No dice :(

[jdef]

mesos-dns likes to use the framework name as part of the domain. On a machine using mesos-dns for resolution, try querying for for nginx-rc.default.pod.kubernetes.mesos

On Fri, Apr 1, 2016 at 12:44 PM, akamalov [email protected] wrote:

@jdef https://github.com/jdef - Hey James. Modified and re-launched km-scheduler with --mesos-generate-task-discovery=true.

Deployed a sample application:

apiVersion: v1 kind: Service metadata: name: nginxsvc labels: app: nginx-http spec: type: NodePort ports:

• port: 80 protocol: TCP name: http selector: app: nginx

  apiVersion: v1 kind: ReplicationController metadata: name: nginx-rc spec: replicas: 1 template: metadata: labels: app: nginx annotations: k8s.mesosphere.io/discovery-name: nginx-rc spec: containers: - name: nginx-http image: nginx ports: - containerPort: 80

Launched application:

[root@node236 k8s]# kubectl create -f nginx-https2.yaml You have exposed your service on an external port on all nodes in your cluster. If you want to expose this service to the external internet, you may need to set up firewall rules for the service port(s) (tcp:30005) to serve traffic.

See http://releases.k8s.io/HEAD/docs/user-guide/services-firewalls.md for more details. service "nginxsvc" created replicationcontroller "nginx-rc" created

Check the status of pods:

[root@node236 k8s]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE NODE nginx-rc-5uqd3 1/1 Running 0 8s node261.local.net [root@node236 k8s]#

Display services:

[root@node236 k8s]# kubectl get services -o wide NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR k8sm-scheduler 172.16.10.161 10251/TCP 1d kubernetes 172.16.10.1 443/TCP 1d nginxsvc 172.16.10.80 nodes 80/TCP 33s app=nginx [root@node236 k8s]#

Try to nslookup the pod - format: ${name}.${pod-namespace}.pod:

[root@node236 k8s]# nslookup nginx-rc-5uqd3.default.pod Server: 127.0.0.1 Address: 127.0.0.1#53

** server can't find nginx-rc-5uqd3.default.pod: NXDOMAIN

[root@node236 k8s]#

No dice :(

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/mesosphere/kubernetes-mesos/issues/125#issuecomment-204466474

[akamalov]

James, thanks a metric ton!! It worked!!!

[jdef]

YW. Be aware that mesos-dns aggressively transforms names. I think there's a recent change in mesos-dns/master that would transform the DI name (for consistency w/ how other labels are transformed) to something like nginx-rc-default-pod.kubernetes.mesos (swapping out '.' for '-' in the DI name).

On Fri, Apr 1, 2016 at 3:32 PM, akamalov [email protected] wrote:

James, thanks a metric ton!! It worked!!!

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/mesosphere/kubernetes-mesos/issues/125#issuecomment-204537042

[akamalov]

Yet again, Thanks so much!!!