Document that consul-k8s-control-plane >=1.0.0 requires gRPC port in Release notes

[bondido]

TODO

We should document that consul-k8s 1.0.x and above requires gRPC port to be enabled in the release notes.

From consul-k8s issue (#1826)

Hi, I run consul-k8s on kubernetes cluster with external (outside of kubernetes) consul servers - just to provide simple k8s services discoverability with consul.

Syncer connects to consul API using HTTP protocol.

After upgrading from version 0.49.2 to version =>1.0.0 consul-k8s-control-plane is no longer able to connect to "consul side".

I know that the parameter that we used until 0.49.2 (-http-addr) is no longer available, so I tried to switch to -addresses and -http-port pair but it seems like syncer ignores -http-port - not trying to use HTTP to connect to consul API and keep using TCP port 0 (Consul servers: addresses=[127.0.0.1:0]).

consul-k8s-control-plane sync-catalog -addresses='127.0.0.1' -http-port='8500' -use-tls=false -to-k8s=false -to-consul=true
[INFO]  consul-server-connection-manager: trying to connect to a Consul server
[INFO]  consul-server-connection-manager: discovered Consul servers: addresses=[127.0.0.1:0]
[INFO]  consul-server-connection-manager: current prioritized list of known Consul servers: addresses=[127.0.0.1:0]
[ERROR] consul-server-connection-manager: connection error: error="failed to switch to Consul server \"127.0.0.1:0\": target sub-connection is not ready (state=TRANSIENT_FAILURE)"

If I provide -grpc-port parameter (and have GRPC enabled on consul servers) - syncer starts connecting to consul OK (Consul servers: addresses=[127.0.0.1:8502])

consul-k8s-control-plane sync-catalog -addresses='127.0.0.1' -http-port='8500' -grpc-port="8502" -use-tls=false -to-k8s=false -to-consul=true
[INFO]  consul-server-connection-manager: trying to connect to a Consul server
[INFO]  consul-server-connection-manager: discovered Consul servers: addresses=[127.0.0.1:8502]
[INFO]  consul-server-connection-manager: current prioritized list of known Consul servers: addresses=[127.0.0.1:8502]
[INFO]  consul-server-connection-manager: connected to Consul server: address=127.0.0.1:8502

Is there any way to force syncer to use HTTP instead of GRPC protocol to connect to consul API? Or maybe I need to switch to GRPC (unfortunately I don't have it turned on by default in my consul servers configuration) ?

Or maybe -http-port should be enough to use HTTP and I should report it as a bug? I didn't find any mention of gRPC requirement in documentation nor in consul-k8s-control-plane sync-catalog --help output.

[david-yu]

Hi @bondido what version of Consul Servers are your external servers running on? You need to upgrade to Consul 1.14.x (which uses GRPC by default) to leverage Consul K8s 1.0.x. We still maintain 0.49.x if you wish to stay on 1.13.x and are not ready to upgrade just yet.

[bondido]

Hi @david-yu , I'm not using consul 1.14 in production yet, but the moment is coming. Anyhow, the tests that I made and logs attached to my first post were done with consul 1.14 (run with -dev flag).

What do you mean saying consul 1.14.x "uses GRPC by default"? According to consul latest version docs ( https://developer.hashicorp.com/consul/docs/agent/config/config-files#grpc_port ) grpc default value is -1 (disabled) but I had it enabled during my tests thanks to -dev flag.

So, is there any way to run consul-k8s-control-plane sync-catalog with HTTP consul API or it's no longer possible since 1.0.x ?

[david-yu]

My mistake, we did change Consul 1.14.x to disable GRPC by default. However this is required for Consul K8s 1.0.x to work as we no longer use HTTP and use only GRPC via https://github.com/hashicorp/consul-server-connection-manager

[david-yu]

I'm going to close this for now, please let us know if you other questions or you feel this should be re-opened.

[bondido]

Thanks @david-yu. So I think the gRPC requirement/HTTP API deprecation should be just stated clearly in docs/release notes.

BTW - what is the purpose of all HTTP API Options and -http-port parameter listed by consul-k8s-control-plane sync-catalog --help ?

[blake]

@bondido As of Consul 1.14, Consul client agents are no longer deployed on Kubernetes. The sync catalog process used to connect to the local HTTP server that was exposed by the client agent. Now that the agent is gone, the sync catalog process should be updated to connect directly to the HTTP API on the Consul servers.

@david-yu I think this issue should be re-opened to track this requirement.

[david-yu]

Hi @bondido sorry for the confusion here. We'll go ahead and add some notes on the release notes for Consul K8s 1.0.x to call this change out.

Catalog sync now requires both HTTP and GRPC to communicate to Consul servers now since we utilize the http connection manager to discover which servers that are healthy using the new gRPC API. The HTTP API for Consul is still utilized for registering services. In your case where the servers are managed outside of Consul K8s, the gRPC port is now required to be open. See https://developer.hashicorp.com/consul/docs/agent/config/config-files#grpc_port and https://developer.hashicorp.com/consul/docs/agent/config/config-files#grpc_tls_port.

[webmutation]

Hello, I recently run into an issue where a deployment from a year ago is running fine with just port 8500, but now on a new deployment with consul image 1.15.2 and chart version 1.1.1 it does not work to connect to the externalServers with just 8500

Can you clarify if gRPC can be disabled altogether in the latest versions of Consul, or if it s a requirement that both http 8500 and gRPC 8502 are open, or is there a way to disable it and use only HTTP