intelmq-manager
intelmq-manager copied to clipboard
certtools
Better error messages for missing authentication
If login is required, but the user is not logged in, the error messages are not good.
On Monitor:
11:21 AM Error loading botnet status: {"errors": {"Authentication Required": "Please provide valid Token verification credentials"}} Unauthorized
On configuration and management page, plus the same of monitor:
11:20 AM Get an error error, Unauthorized when trying to obtain config file properly http://example.com/intelmq/v1/api/config?file=runtime 11:20 AM Error incomplete botnet: {"errors": {"Authentication Required": "Please provide valid Token verification credentials"}} Unauthorized
On check page: pop-up with error getting check command output
Ob about: pop-up with Error getting debugging information. Do you have IntelMQ >= 2.2.0?
10:45 AM 24× Error loading bot queues information: {"errors": {"Invalid Authentication": "Provided Token verification credentials were invalid"}} Unauthorized
10:44 AM Get an error error, Unauthorized when trying to obtain config file properly http://172.29.26.232/intelmq/v1/api/config?file=positions .
10:44 AM Get an error error, Unauthorized when trying to obtain config file properly http://172.29.26.232/intelmq/v1/api/config?file=pipeline .
10:44 AM Get an error error, Unauthorized when trying to obtain config file properly http://172.29.26.232/intelmq/v1/api/config?file=runtime .
10:44 AM Get an error error, Unauthorized when trying to obtain config file properly http://172.29.26.232/intelmq/v1/api/config?file=defaults .
10:44 AM Get an error error, Unauthorized when trying to obtain config file properly http://172.29.26.232/intelmq/v1/api/config?file=bots .
Hello,
Sorry to up this topic but I've the same issue. Any help ?
Thank in advance :)
For any help more information is required: What error messages do you get at what action and which preconditions (e.g. logged in), how did you install and set up IntelMQ, which operating system are you using etc.
@aragorne007 It could be that it is redis that is giving you a problem solution:
chown -R redis:redis /var/log/redis chmod -R u+rwX,g+rwX,u+rx /var/log/redis chmod +r /etc/redis/redis.conf systemctl start redis
@oencarnacion I tried but it doesn't work.
@sebix I installed intelmq, intelmq-manager and intelmq-api. After that, I've created an account with the cmd "intelmq-api-adduser".
When I'm log in and when I go to any menu, I've 2 kinds of error :
First One (Configuration & Monitor) :
22:27:55 369× Error loading bot queues information: {"errors": {"Invalid Authentication": "Provided Token verification credentials were invalid"}} Unauthorized
22:21:53 Get an error error, Unauthorized when trying to obtain config file properly http://192.168.0.21/intelmq/v1/api/positions .
22:21:53 Get an error error, Unauthorized when trying to obtain config file properly http://192.168.0.21/intelmq/v1/api/runtime .
22:21:53 Get an error error, Unauthorized when trying to obtain config file properly http://192.168.0.21/intelmq/v1/api/bots .
The second one (Check) :
error getting check command output
I'm sure that it's a permission issue but I'm not sure to understand which folder need to have the right access. Or maybe it's a file configuration error but I think that I've well followed the Installation guide line
Thank for your help :)
If it can help, the result of "intelmqctl debug"
Paths: HARMONIZATION_CONF_FILE: '/etc/intelmq/harmonization.conf' RUNTIME_CONF_FILE: '/etc/intelmq/runtime.yaml' VAR_RUN_PATH: '/var/run/intelmq/' STATE_FILE_PATH: '/var/lib/intelmq/state.json' DEFAULT_LOGGING_PATH: '/var/log/intelmq/' file: '/usr/lib/python3/dist-packages/intelmq/bin/intelmqctl.py' CONFIG_DIR: '/etc/intelmq/' ROOT_DIR: '/' Environment variables: INTELMQ_ROOT_DIR: None INTELMQ_PATHS_NO_OPT: None INTELMQ_PATHS_OPT: None INTELMQ_MANAGER_CONTROLLER_CMD: None PATH: '/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games'
I assume you installed the deb/rpm packages? Which operating system?
When I'm log in and when I go to any menu, I've 2 kinds of error :
Is the login successful (Is the username shown instead of the login-button after the login?) I assume it is, because then the error message would be different.
What you are seeing must be some glitch in the authentication between the manager and the API.
See also https://intelmq.readthedocs.io/en/latest/user/intelmq-api.html#access-denied-authentication-required-please-provide-valid-token-verification-credentials (but that does not really apply as you are using the API indirectly with the Manager). The Manager needs to pass the token to the API in its calls.
If you are up for some debugging, check what you see in the HTTP request headers of the API calls (using the debug tools of the browser).
@sebix
The documentation isn't enough complete I think :(
I'm not enough documented to try some debug with HTTP request
In "check" panel, I've this error : error getting check command output
In "about" panel, I've 2 others error messages : Error getting debugging information. Do you have IntelMQ >= 2.2.0?
error getting version
I don't know if it can help you or give you some clue ?
In order to reconstruct your error, it is still important to know if you installed the deb/rpm packages or via Pip or from git, and which operating system you are using?
The documentation isn't enough complete I think :(
Yes, I know. That section is not meant for the error you are experiencing, I linked it because it is related.
I'm not enough documented to try some debug with HTTP request
Which browser are you using? I can guide you.
In "check" panel, I've this error : error getting check command output
In "about" panel, I've 2 others error messages : Error getting debugging information. Do you have IntelMQ >= 2.2.0?
error getting version
All that indicates an error with the authentication, but no trace on the reasons.
I don't know if it can help you or give you some clue ?
I have no clue what the error is, either, but I am offering some of my spare time so we can debug it together.
Hi @aragorne007 confirm me that in this /var/lib/ path you have a folder called dbconfig-common
Hello @sebix ;
I installed the package from pip3 install.
I'm using Google Chrome :)
Yes, It's what I expect, an authentication issue but I didn't find any log to prove that.
Hello @oencarnacion,
Yes, and "intelmq" user is the owner BUT I think that I've manually created this folder because it wasn't create during the installation. That can helping you ?
Thank a lot for you help guys :)
@aragorne007 The solution to your problem is to go to the /var/lib/ path, unzip the zip that I supply, give it read and write permission and then proceed to create the user and if you want, restart the server just in case and that's it, it should work for you. intelmq bug both for installation of binaries or through python.
You know @sebix :)
@oencarnacion Thank a lot for your help :)
I've move your folder to my IntelMQ server and, to be sure, add chmod 766 :P But, when I try to create a new IntelMQ account, I've this error :
Loading config from /etc/intelmq/api-config.json
Traceback (most recent call last):
File "/usr/local/bin/intelmq-api-adduser", line 27, in
I guess, it's a new permission issue :P
What is the service used by intelmq ? To know which service restart to apply new modification
Ohh no, I find the issue :
cat /etc/intelmq/api-config.json { "intelmq_ctl_cmd": ["sudo", "-u", "intelmq", "intelmqctl"], "allowed_path": "/var/lib/intelmq/bots", "session_store": "/var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi", "session_duration": 86400, "allow_origins": ["*"] }
The session_store folder doesn't exists
/var/lib/dbconfig-common/ └── dbconfig-common ├── config ├── intelmq-api.conf └── sqlite3 └── intelmq-api
Can I create the folder myself ?
I installed the package from pip3 install.
OK, then it's clear that some manual work needs to be done. Installing via pip is more difficult than the automated deb/rpm package installation.
Permissions 766 don't sound right at all. Make the file and the directory containing the file writable for the webserver, only the webserver.
What is the service used by intelmq ? To know which service restart to apply new modification
The API is run by apache -> sudo systemctl restart apache2
I'm dumb ^^ The folder wasn't /var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi but /var/lib/dbconfig-common/dbconfig-common/sqlite3/intelmq-api/intelmqapi.
So twice dbconfig-common in depth ^^ I fixed that.
I created the new user with intelmq-api-adduser succesfully but now, in the intelmq-manager panel, I can't login. I've this error :
Login failed with unknown reason. Please report this bug.
Redis and Apache2 has ben restarted
[Wed Aug 17 22:32:41.602163 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] mod_wsgi (pid=941): Exception occurred processing WSGI script '/usr/lib/python3/dist-packages/intelmq_api/intelmq-api.wsgi'., referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602322 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] Traceback (most recent call last):, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602385 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/lib/python3/dist-packages/intelmq_api/intelmq-api.wsgi", line 12, in application, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602390 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] return __hug_wsgi__(environ, start_response), referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602399 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/hug/api.py", line 500, in api_auto_instantiate, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602403 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] module.__hug_wsgi__ = module.__hug__.http.server(), referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602412 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/hug/api.py", line 371, in server, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602416 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] self.api._ensure_started(), referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602425 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/hug/api.py", line 638, in _ensure_started, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602429 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] startup_handler(self), referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602437 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/intelmq_api/serve.py", line 46, in setup, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602441 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] intelmq_api.api.initialize_api(api_config), referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602450 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/intelmq_api/api.py", line 82, in initialize_api, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602454 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] session_store = session.SessionStore(str(session_file),, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602463 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/intelmq_api/session.py", line 86, in __init__, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602467 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] self.init_sqlite_db(), referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602476 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/intelmq_api/session.py", line 100, in init_sqlite_db, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602480 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] with self.connect() as con:, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602488 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] File "/usr/local/lib/python3.9/dist-packages/intelmq_api/session.py", line 91, in connect, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602492 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] return sqlite3.connect(self.dbname, check_same_thread=False,, referer: http://192.168.0.21/intelmq-manager/
[Wed Aug 17 22:32:41.602512 2022] [wsgi:error] [pid 941:tid 140627228423936] [client 192.168.0.144:59501] sqlite3.OperationalError: unable to open database file, referer: http://192.168.0.21/intelmq-manager/
The log for 1 attemp
It look like another permission missing
is the session database and the directory containing the file writable for the webserver? Please show jq .session_store /etc/intelmq/api-config.json | xargs ls -l and jq .session_store /etc/intelmq/api-config.json | xargs dirname | xargs ls -ld
root@HostIntell:/var/lib# jq .session_store /etc/intelmq/api-config.json | xargs ls -l
-rw-r--r-- 1 root root 24576 Aug 17 22:30 /var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi
root@HostIntell:/var/lib# jq .session_store /etc/intelmq/api-config.json | xargs dirname | xargs ls -ld
drwxrw-rw- 2 intelmq intelmq 4096 Aug 17 22:30 /var/lib/dbconfig-common/sqlite3/intelmq-api
root@HostIntell:/var/lib#
"root" for the first one. I'll change that. Can you confirm that the user need to be used is "intelmq" ? Because it's the user I used eveywhere in the system (not the user used in the webGUI )
Can you confirm that the user need to be used is "intelmq" ?
I assume the user your webserver is using, is not intelmq, but rather apache, www-data or similar. Use that one.
Can you confirm that the user need to be used is "intelmq" ?
I assume the user your webserver is using, is not intelmq, but rather apache, www-data or similar. Use that one.
Ohhh, It's maybe that the main issue. Every intelmq folder is "own" by "intelmq" user and not apache or www-data
Do you have an exhaustive list of all folder that I need to change the owner from "intelmq" to "www-data" ?
www-data is the user mention and existing in the /etc/passwd file. "Apache" is not existing
No, because this command isn't existing. Maybe another issue with the pip3 install ?
I performed "intelmqctl check" before and "No Issues Found" diplayed.
Did you use https://intelmq.readthedocs.io/en/maintenance/user/installation.html#pypi or a different installation method?
Yes, I used this one.
I tried a new time :
root@HostIntell:/var/lib# sudo -i
root@HostIntell:~# pip3 install intelmq
Requirement already satisfied: intelmq in /usr/lib/python3/dist-packages (3.0.2)
root@HostIntell:~# useradd -d /opt/intelmq -U -s /bin/bash intelmq
useradd: user 'intelmq' already exists
root@HostIntell:~# sudo intelmqsetup
sudo: intelmqsetup: command not found
root@HostIntell:~#
I tried also that, but not really relevant :
root@HostIntell:/var/lib# apt install intelmqsetup
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package intelmqsetup
root@HostIntell:/var/lib# apt search intelmqsetup
Sorting... Done
Full Text Search... Done
root@HostIntell:/var/lib# apt search intelmq
Sorting... Done
Full Text Search... Done
intelmq/unknown,now 3.0.2-1 all [installed]
Solution for IT security teams for collecting and processing security feeds
intelmq-api/unknown,now 3.0.1-1 all [installed,automatic]
HUG based API for the intelmq project
intelmq-manager/unknown,now 3.0.1-1 all [installed]
Graphical interface to manage configurations for the IntelMQ framework.