[Q]Tag Setup with Graphite-web

[mmueller101]

I was looking for a better understanding and some help with setting up tags in go-carbon with a Graphite-web front end. Currently, I have been unable to get Graphite-web to properly query tags that have been sent to go-carbon.

I see that when you send a metric with a tag, the tag gets stored in the _tagged directory, but the metric itself never gets created or updated with the value sent. The value looks to be stored in a tag file created for the tag. Is this the intended behavior? How does the tag file get updated when there is a new metric associated with a tag? Are there performance hits when more metrics are associated with a tag?

Also, are there consideration to allowing alternative methods of storing tags? i.e. sending tag data to redis? Tags would be a great way to keep track of metrics from sources that may not last for more than a week or two. Having tags pointing to old instances would not be ideal since they no longer exist.

Thanks.

[deniszh]

Hi @mmueller101 ,

go-carbon tags support currently limited:

1. external TagDB is mandatory. I.e. you need separate graphite-web, preferrably with TagDB Redis backend configured. Then you need to point go-carbon to that tagdb instance, using tagdb-url parameter in [tags] section. (ofc tag support should be enabled too - by default tags.enabled = false).
2. carbonserver currently do not support tags. So, you can't set go-carbon as CLUSTER_SERVER to graphite-web, you can only use it normally, sharing whisper files with graphite-web and using carbonlink for cache calls (which enabled by default). Some incomplete implementation on carbonserver tags are located in https://github.com/Civil/go-carbon-old/tree/carbonserver-tags but it will require some work. /cc @Civil

[deniszh]

I.e. if you need working tag support in Graphite you can currently use either

1. graphite-web with Redis TagDB and normal carbon, e.g. in https://github.com/graphite-project/docker-graphite-statsd It will not give you great performance, though - but will work. either
2. graphite-clickhouse + carbon-clickhouse + clickhouse + carbonapi - this setup is much more complicated then single docker image, though - although scalable and prod ready. You can check e.g. https://github.com/lomik/graphite-clickhouse-tldr either
3. victoriametrics + carbonapi - that setup also should work but kinda experimental. See https://github.com/deniszh/graphite-victoriametrics-tldr

[Civil]

@deniszh Yeah, I've never finished integrating @bom-d-van's PR to go-carbon and finishing boilerplate. I still plan to do that though (AFAIR it's based on go-carbon 0.13 and it implements working tag support for find queries, for simple requests, but requires same boilerplate for render and other stuff)

[bom-d-van]

hi, I can see if I could also channel some effort back to the tag support as well.

I'm a bit biased on using tags. I think having properly designed hierarchy for metrics would make things much better.

Graphite has been around for a long time. But its original design ages quite well IMO: file-tree-like metric hierarchy, proper retention schema for individual metrics, etc.

For no reason, I just noticed that in OpenTSDB they actually support using tags to build a metric tree for easy navigation: http://opentsdb.net/docs/build/html/user_guide/trees.html

[Civil]

Problem with any tags support on go-carbon side, that there should be some sort of compatibility with the way how it's done in graphite-web as people would expect it to be compatible (at least by default). And I still think that the design for the tag support that was done back in 2017 I think would perform and scale better than what graphite-web suggests.

[mmueller101]

Sorry for the delayed response. This was very helpful in understanding how tagging is being implemented. The issue was the carbon user did not have permissions for the 'local-dir' specified. Sorry for the miss. Thanks for the quick responses.