401 returned when using plugin - even though standard Basic Auth curls work using creds

[mmpashby]

I am using Python 2.7.3

Collectd config looks like this:-

TypesDB "/usr/share/collectd/types.db" LoadPlugin python <Plugin python> LogTraces true Interactive false Import "collectd_rabbitmq.collectd_plugin" <Module "collectd_rabbitmq.collectd_plugin"> Username "<username>" Password "<password>" Realm "RabbitMQ Management" Host "localhost" Port "15672" </Module> </Plugin>

My Error's look like this in the collectd log:-

[2016-04-25 15:09:10] HTTP Error: HTTP Error 401: Unauthorized [2016-04-25 15:09:10] Unhandled python exception in read callback: TypeError: 'NoneType' object is not iterable [2016-04-25 15:09:10] Traceback (most recent call last): [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 89, in read PLUGIN.read() [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 114, in read self.dispatch_exchanges(vhost_name) [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 182, in dispatch_exchanges stats = self.rabbit.get_exchange_stats(vhost_name=vhost_name) [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 135, in get_exchange_stats return self.get_stats('exchange', exchange_name, vhost_name) [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 180, in get_stats names = stat_name_func(vhost) [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 126, in get_exchange_names return self.get_names(all_exchanges) [2016-04-25 15:09:10] File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 43, in get_names for item in items: [2016-04-25 15:09:10] TypeError: 'NoneType' object is not iterable [2016-04-25 15:09:10] read-function of pluginpython.collectd_rabbitmq.collectd_plugin' failed. Will suspend it for 160.000 seconds.`

If I use similar urllib2 code as a test my api call works fine:-

import urllib2 SERVER = 'localhost:15672' authinfo = urllib2.HTTPPasswordMgrWithDefaultRealm() authinfo.add_password('RabbitMQ Management', SERVER, '', '') page = 'HTTP://'+SERVER+'/api/vhosts' handler = urllib2.HTTPBasicAuthHandler(authinfo) myopener = urllib2.build_opener(handler) opened = urllib2.install_opener(myopener) output = urllib2.urlopen(page) print output.read()

Whats strange about the exceptions is that even though they happen, I eventually have data in my graphite installation but can be delayed significantly.

[jimbydamonk]

Hmm. Is there anything in the RabbitMQ logs to indicate an authorization error when calling the API? Does the user that you are using have the proper permissions on all vhosts?

[mrunge]

I hit the same issue on CentOS 7, with SELinux set to enforcing.

type=AVC msg=audit(1463997426.206:7526): avc: denied { name_connect } for pid=10268 comm="collectd" dest=15672 scontext=system_u:system_r:collectd_t:s0 tcontext=system_u:object_r:amqp_port_t:s0 tclass=tcp_socket

A quick check via setenforce 0 shows, it works correctly (after figuring out, I had to enable rabbitmq_management plugin.

setsebool collectd_tcp_network_connect 1 Fixed this for me.

[jimbydamonk]

@Pashbee It has been a while since you posted this issue. Have you tried @mrunge suggestion ?

[janboll]

The following worked for me: https://github.com/NYTimes/collectd-rabbitmq/pull/50

[hadret]

Still hitting same problem; using latest release, 1.17.0. curl with same credentials is working fine from the console on the given host, same goes for simple python script compatible with statsd.

collectd-rabbitmq:1.17.0 (installed via pip) collectd:5.6.1.30 (installed via APT, ordinary debian package) OS:Ubuntu 14.04 LTS

Error messages:

Apr 12 14:00:39 $HOST collectd[3670]: HTTP Error: HTTP Error 401: Unauthorized
Apr 12 14:00:39 $HOST collectd[3670]: Unhandled python exception in read callback: TypeError: 'NoneType' object is not iterable
Apr 12 14:00:39 $HOST collectd[3670]: Traceback (most recent call last):
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 90, in read
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 127, in read
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 267, in dispatch_exchanges
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 135, in get_exchange_stats
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 186, in get_stats
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 126, in get_exchange_names
Apr 12 14:00:39 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 43, in get_names
Apr 12 14:00:39 $HOST collectd[3670]: TypeError: 'NoneType' object is not iterable
Apr 12 14:00:39 $HOST collectd[3670]: read-function of plugin `python.collectd_rabbitmq.collectd_plugin' failed. Will suspend it for 20.000 seconds.
Apr 12 14:00:59 $HOST collectd[3670]: HTTP Error: HTTP Error 401: Unauthorized
Apr 12 14:00:59 $HOST collectd[3670]: Unhandled python exception in read callback: TypeError: 'NoneType' object is not iterable
Apr 12 14:00:59 $HOST collectd[3670]: Traceback (most recent call last):
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 90, in read
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 127, in read
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/collectd_plugin.py", line 267, in dispatch_exchanges
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 135, in get_exchange_stats
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 186, in get_stats
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 126, in get_exchange_names
Apr 12 14:00:59 $HOST collectd[3670]:   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq/rabbit.py", line 43, in get_names
Apr 12 14:00:59 $HOST collectd[3670]: TypeError: 'NoneType' object is not iterable
Apr 12 14:00:59 $HOST collectd[3670]: read-function of plugin `python.collectd_rabbitmq.collectd_plugin' failed. Will suspend it for 40.000 seconds.

Config:

TypesDB "/usr/local/share/collectd-rabbitmq/types.db.custom"

LoadPlugin python
<Plugin python>
  LogTraces true
  Interactive false
  Import "collectd_rabbitmq.collectd_plugin"
  <Module "collectd_rabbitmq.collectd_plugin">

    Username "$USER"
    Password "$PASSWORD"
    Realm "RabbitMQ Management"
    Host "localhost"
    Port "15672"
  </Module>
</Plugin>

[jimbydamonk]

I just booted the vagrant image with this repo and changed the password that I was using in the config. got a different error than you did.

root@collectd-rabbitmq:~# tail -f /var/log/collectd.log 
[2017-04-20 15:41:32] URL Error: <urlopen error [Errno 111] Connection refused>
[2017-04-20 15:41:32] URL Error: <urlopen error [Errno 111] Connection refused>
[2017-04-20 15:58:58] Exiting normally.
[2017-04-20 15:58:58] collectd: Stopping 5 read threads.
[2017-04-20 15:58:58] collectd: Stopping 5 write threads.
[2017-04-20 15:58:58] supervised by systemd, will signal readyness
[2017-04-20 15:58:58] Initialization complete, entering read-loop.
[2017-04-20 15:58:58] HTTP Error: HTTP Error 401: Unauthorized

and on the rabbitmq side a series of

=ERROR REPORT==== 20-Apr-2017::16:01:28 ===
webmachine error: path="/api/nodes"
"Unauthorized"

=WARNING REPORT==== 20-Apr-2017::16:01:28 ===
HTTP access denied: user 'collectd' - invalid credentials

What does your rabbitmq log look like ?

[hadret]

Same -- it was lacking access for one of the API calls for the user used to draw the graphs, which was fatal for the plugin to start. Either way -- I bumped up the plugin to latest 1.18.0, fixed the permissions for the user and now all is working as expected! :+1:

[azhurbilo]

May be check that your user have access to "/" virtualhost to iterate virtual-hosts https://github.com/NYTimes/collectd-rabbitmq/blob/v1.18.0/collectd_rabbitmq/collectd_plugin.py#L126

[tnibert]

I encountered this issue as well on Debian Jessie. Fixed by skipping the / vhost in CollectdPlugin.read(). Submitted pull request, but may not address underlying issue.

[abompard]

I may have fixed this with PR #74 .

[sylvainfaivre]

If you have this problem, check your RabbitMQ logs. I had the same error when using a normal RabbitMQ user, and the logs said :

2020-03-09 16:56:26.146 [warning] <0.5177.31> HTTP access denied: user 'user' - Not monitor user

I solved it by giving the monitoring rights to the user with this command :

rabbitmqctl set_user_tags <user> monitoring

Then I ran into other errors, but that's for another issue.