nsqadmin: does not properly prefix statsd urls

[ambarket]

I'm a newbie to nsq, graphite, and statsd so I apologize if this is a trivial oversight on my part. I tried to search through the code for where these urls are being generated but I couldn't locate it. Thanks in advance for taking a look at this.

I'm running on nsq 0.3.7 (precompiled binaries). The server is a recently fresh install of Ubuntu 14.04. I just installed graphite, carbon, and statsd using the tutorials at digital ocean (https://www.digitalocean.com/community/tutorials/how-to-install-and-use-graphite-on-an-ubuntu-14-04-server)

I can see that all the data is properly published to statsd by nsqd because I can view the graphs using the graphite browser. However the only graphs that show up correctly in the nsqadmin panel are depth graphs for particular channels.

Here is the url generated for the garbage collection summary of my server.

http://mysite.com/render?colorList=red%2Cgreen%2Cblue%2Cpurple&height=450&width=800&bgcolor=ff000000&fgcolor=999999&yMin=0&lineMode=connected&drawNullAsZero=false&from=-1h&until=-1min&target%5B%5D=summarize(nsq.ServerName.mem.*_bytes%2C%2260sec%22%2C%22avg%22)

In this case, manually appending the stats.gauges prefix makes it work.

...summarize(**stats.gauges.**nsq.ServerName.mem.*_bytes...

For a garbage collection counters, I also have to append ".count" to the end of the statsd url. ...summarize(scale(movingAverage(stats.counters.nsq.ServerName.mem.gc_runs.count...

For a topic depth graph, it is placing "undefined" as the topic name in the url, but the statd prefix is correctly appended. ...sumSeries(stats.gauges.nsq.ServerName.topic.undefined.depth)...

Yet the urls for depths of particular channels on a topic are correctly generated, and the graphs for number of connections to a channel are also correct.

Also the RATE column on the topics page simply displays the text "ERROR".

The counter page of nsqadmin is a blank white screen when I have graphs enabled. In the javascritp consol, the following error is logged.

"main.js:3026 - Uncaught ReferenceError: prefix is not defined"

By the way NSQ is awesome, great work on building this system. Thanks!

[ploxiln]

I set --use-statsd-prefixes=false and also configure my statsd implementation to not include such prefixes. It's possible that if you do have/want such prefixes, it's less tested and buggy.

It may help to mess with the nsqadmin statsd prefix options

$ nsqadmin --help
Usage of nsqadmin:
...
  -statsd-counter-format string
        The counter stats key formatting applied by the implementation of statsd. If no formatting is desired, set this to an empty string. (default "stats.counters.%s.count")
  -statsd-gauge-format string
        The gauge stats key formatting applied by the implementation of statsd. If no formatting is desired, set this to an empty string. (default "stats.gauges.%s")
  -statsd-prefix string
        prefix used for keys sent to statsd (%s for host replacement, must match nsqd) (default "nsq.%s")
  -use-statsd-prefixes
        (Deprecated - Use --statsd-counter-format and --statsd-gauge-format) Expect statsd prefixed keys in graphite (ie: 'stats.counters.' and 'stats.gauges.') (default true)
...

re: rate column errors: see #734

One last tip: your issue title is way too long, try to shorten :)

[ambarket]

The workaround was to switch to using bitly's statsdaemon rather than the original statsd. As mentioned in the docs, statsdaemon by default does not prefix anything. I tried messing with the prefix command line options and couldn't get it to work with statsd.

It would be nice to have full support for statsd's new standard way of organizing the metrics because I do like how they are laid out. Though for now, I'll just use statsdaemon, which I assume is what most developers on this project are using.

[mreiferson]

You probably landed on at least a bug or two here, regardless. We intend to support any combination of statsd prefixing, so we should look into it.

Thanks for the report.

[mreiferson]

Btw, can you paste the original arguments you used to run both nsqd and nsqadmin when things weren't working?

[ambarket]

Yeah here are the commands.

nsqd -statsd-address=domain.com:8125 -data-path=/path/to/nsq-message-queues/ -broadcast-address=host.domain.com -tcp-address=0.0.0.0:6042 -http-address=0.0.0.0:6043 -lookupd-tcp-address=host.domain.com:6044

nsqadmin -graphite-url=http://domain.com:6049 -http-address=0.0.0.0:6046 -lookupd-http-address=host.domain.com:6045

nsqlookupd -tcp-address=0.0.0.0:6044 -http-address=0.0.0.0:6045

The only change I made to make it work with statsdaemon was to add the -use-statsd-prefixes=false flag to nsqadmin

Also, even though the graphs are working now. The counter page still crashes with the error "main.js:3026 - Uncaught ReferenceError: prefix is not defined" when the graphs are enabled. If I set the graph timeframe to "off", the counter page begins working again without restarting the processes. Maybe this should be another issue?