Bosh-DNS names as destination Application Security Group

[kinjelom]

Issue

I want to use Bosh-DNS names as a destination in an Application Security Group definition. Example:

[
  {
    "destination": "q-s0.postgres-instance.default.postgres-deployment.bosh",
     "ports": "5432",
     "protocol": "tcp"
  }
]

Now I can use it in user-provided-services definition, but ASG destination doesn't support host names.

Context

One source of truth is usually a good idea. I think BOSH-DNS is source of truth about IP addresses, so why don't use it in an Application Security Group definition. BTW, until openstack will not support local static IP (https://github.com/cloudfoundry/bosh-openstack-cpi-release/issues/14), using BOSH-DNS names can be walkaround. Local IP addresses can be changed automatically by bosh-director... then ASG definitions have outdated information.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/168481622

The labels on this github issue will be updated when the story is started.

[cwlbraa]

This is a super interesting request that'd involve a bunch of different system components. I think, as it stands, Diego takes advantage of how static these are and uses them to program garden's allows/disallows for each container. With a DNS name,You might need to resolve the names repeatedly in order to set up firewall rules way down there on each cell, but I'm really no expert. Silk is sort of in this business for container overlay networks, but they use virtual IPs.

@sunjayBhatia or @rosenhouse might find the "hows" of this feature request interesting.

[rosenhouse]

cc @adobley @KauzClay @ameowlia @shalako who have been thinking about enhancements to ASGs

[kinjelom]

@cwlbraa Maybe something like a asg-resolver-worker can watch security groups entries based on hostname and update the destination IPs (update-security-group) every time change occurs? Should it cause a restart/restage of applications?

In that case th ASG definition would look like this:

[
  {
     "destination-hostname": "q-s0.postgres-instance.default.postgres-deployment.bosh",
     "destination": "10.0.1.10,10.0.1.11,10.0.1.12",
     "ports": "5432",
     "protocol": "tcp"
  }
]

[cwlbraa]

@wildloop that strategy definitely wouldn't need a restage, but would probably necessitate a restart. The fact that that restart would cause downtime means that a changing A-record could take down an app temporarily, which seems like a non-starter, especially given that bosh-dns & consul dynamically change A records all the time based on the health of services.