Consul service registration from gocast after BGP route announcement as sot health check for dependent app using the tags

[pponakan]

This PR introduces a new health check registered by gocast to the local consul agent after it has announced the route for the VIP requested by the dependent service running in nomad. Gocast periodically discovers services that request VIP announcements via consul tags in the nomad spec. After reading the tags, gocast will now register a new service check indicating the BGP route + dependent app's health. The new sservice name can be set through additional gocast tags, or defaults to the dependent service name with "-vip" suffix. Example tags:

  service {
    name = "some-service"

    tags = [
      "enable_gocast",
      "gocast_vip=${ALLOC_PUBLICVIP}",
      "gocast_monitor=consul",
      "gocast_consul_vip_service=some-vip-service",
      "gocast_consul_vip_check=port:tcp:3128",
      "gocast_consul_vip_check=interval:5s",
      "site=ams1",
    ]

Gocast required write permissions to be able to register or deregister a new health check. The service name may need to be whitelisted depending upon the consul ACL token in the cluster where the node joins to.

The above functionality works as intended. However, 2 issues were noticed while testing. During some of the periodic discover req's to the local consul agent it is occasionally returned no data as the ?stale flag is set in the request. When consul is in the problematic state, removing the stale flag in the catalog req returns all local services.

As there is no data from consul. gocast starts to withdraw the route for all active VIP's. During my testing, I had 2 VIP's injected by gocast on the local node and abut 75% of the time both would get withdrawn (along with the associated new service health check). A few times, it gets deadlocked after withdrawing one of the VIP's in the monitor Remove() func.