consul-template icon indicating copy to clipboard operation
consul-template copied to clipboard

Published 20 hours ago • verified publisher icon hashicorp

Allow setting dedup prefix on cli

Open nosammai opened this issue 3 years ago • 5 comments

Allows for setting the dedup prefix on the command line. This can be nice if you want to set different prefixes at runtime without having to template the consul-template config file.

nosammai avatar May 03 '21 18:05 nosammai

CLA assistant check
All committers have signed the CLA.

hashicorp-cla avatar May 03 '21 18:05 hashicorp-cla

Hey @nosammai, thanks for the PR. Before reviewing this I have a question...

Have you tested and/or do you plan on switching to Consul's health streaming support? This is a beta/experimental feature in the Consul 1.9 series but will be on by default in 1.10. I'm hoping it will make dedup unnecessary and I'm hoping to remove it if this is the case.

Thanks.

eikenb avatar May 14 '21 23:05 eikenb

I haven't tested it yet, but definitely plan on switching over to that when it's no longer experimental. If it resolves our performance issues, we would like to use that instead of dedup on consul-template.

Does the service API function in consul-template use the Health API that has streaming enabled?

nosammai avatar May 15 '21 00:05 nosammai

Does the service API function in consul-template use the Health API that has streaming enabled?

Yes, as long as the consul servers and the local consul agent all support the streaming protocol (all are 1.10+). Consul-template uses the local agent for queries and agent-server communications that support it [1] will get streaming automatically.

[1] this is just the health related service information for now

eikenb avatar May 15 '21 00:05 eikenb

Marked this with "thinking" to note that it's waiting until after the 1.10 streaming is out to see how much it helps with the network performance overhead which people mitigate using dedup.

eikenb avatar May 15 '21 00:05 eikenb

One important thing that the dedup functionality helps with is when we need to watch 1000+ services and we do not want to do that on every instance of consul-template. Would streaming help with that as well?

komapa avatar Sep 29 '22 19:09 komapa

@nosammai were you ever able to test this out? and how did it go?

Is this still usable or can this PR be close now?

roncodingenthusiast avatar Jun 21 '23 17:06 roncodingenthusiast

Yep the streaming functionality fixed the issues we were having that required us to use dedupe.

nosammai avatar Jun 22 '23 00:06 nosammai

@nosammai great, thank you for getting back to me. Will be closing this PR now as not needed anymore since the streaming functionality helped fix the issues you were having that required the use of dedup

roncodingenthusiast avatar Jun 22 '23 12:06 roncodingenthusiast