Pods DNS doesn't allow resolution of bosh-dns aliases

[bstick12]

We are deploying a simple HAProxy container as a DaemonSet.

DockerFile

FROM haproxy:1.7

RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y dnsutils

COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg

haproxy.cfg

global
  maxconn 64000
  spread-checks 4

defaults
  timeout connect 5000ms
  timeout client 50000ms
  timeout server 50000ms

listen master-nodes
  bind *:8443
  mode tcp
  balance roundrobin
  server masters master.cfcr.internal

The pods go into a crash backoff loop. The logs from the pods are as follows.

<7>haproxy-systemd-wrapper: executing /usr/local/sbin/haproxy -p /run/haproxy.pid -db -f /usr/local/etc/haproxy/haproxy.cfg -Ds
[ALERT] 150/091531 (9) : parsing [/usr/local/etc/haproxy/haproxy.cfg:14] : 'server masters' : could not resolve address 'master.cfcr.internal'.
[ALERT] 150/091531 (9) : Failed to initialize server(s) addr.
<5>haproxy-systemd-wrapper: exit, haproxy RC=1

According to https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#inheriting-dns-from-the-node the dns should be inherited from the worker node. Which is able to resolve master.cfcr.internal

Expected Behavior:

• When I deploy the pod above I should be able to resolve master.cfcr.internal or any other bosh-dns address

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/157994996

The labels on this github issue will be updated when the story is started.

[tvs]

Heapster is using master.cfcr.internal to reference the API server, so there's no reason it shouldn't work elsewhere: https://github.com/cloudfoundry-incubator/kubo-release/blob/master/jobs/apply-specs/templates/specs/heapster.yml.erb#L62

Had you run the apply-specs errand to get kube-dns up and running?

edit:

Just to check against my own cluster, I exec'd into one of my running containers and was able to ping it:

# kubectl exec -it blog-pod -c hugo -- /bin/sh
/src # ping master.cfcr.internal
PING master.cfcr.internal (10.0.48.4): 56 data bytes
64 bytes from 10.0.48.4: seq=0 ttl=63 time=1.404 ms
64 bytes from 10.0.48.4: seq=1 ttl=63 time=0.382 ms
^C
--- master.cfcr.internal ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.382/0.893/1.404 ms

This is with a dev build of what eventually became v0.17.0.

[addisonhuddy]

Closing out due to inactivity.

[akshaymankar]

@addisonhuddy There was activity on tracker for this bug. This bug still exits and affects telemetry components. We would like to get this fixed before next release. We can help you reproduce this issue if required.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/162158342

The labels on this github issue will be updated when the story is started.