icinga2
icinga2 copied to clipboard
Icinga
Icinga2 crash: __cxa_throw in /usr/lib64/icinga2/sbin/icinga2
Describe the bug
Icinga has crashed:
[2022-02-05 19:00:37 +0100] information/FileLogger: 'main-log' started.
[2022-02-05 19:00:37 +0100] information/ApiListener: 'api' started.
[2022-02-05 19:00:37 +0100] information/ApiListener: Started new listener on '[::]:5665'
[2022-02-05 19:00:37 +0100] critical/cli: Exception in main process: Error: basic_ios::clear
Stacktrace:
0# __cxa_throw in /usr/lib64/icinga2/sbin/icinga2
1# std::__throw_ios_failure(char const*) in /lib64/libstdc++.so.6
2# 0x00007F520F2B1F6A in /lib64/libstdc++.so.6
3# icinga::Utility::SaveJsonFile(icinga::String const&, int, icinga::Value const&) in /usr/lib64/icinga2/sbin/icinga2
4# icinga::ApiListener::UpdateStatusFile(boost::asio::ip::basic_endpoint<boost::asio::ip::tcp>) in /usr/lib64/icinga2/sbin/icinga2
5# icinga::ApiListener::AddListener(icinga::String const&, icinga::String const&) in /usr/lib64/icinga2/sbin/icinga2
6# icinga::ApiListener::Start(bool) in /usr/lib64/icinga2/sbin/icinga2
7# icinga::ConfigObject::Activate(bool, icinga::Value const&) in /usr/lib64/icinga2/sbin/icinga2
8# icinga::ConfigItem::ActivateItems(std::vector<boost::intrusive_ptr<icinga::ConfigItem>, std::allocator<boost::intrusive_ptr<icinga::ConfigItem> > > const&, bool, bool, bool, icinga::Value const&) in /usr/lib64/icinga2/sbin/icinga2
9# 0x0000000000B81C00 in /usr/lib64/icinga2/sbin/icinga2
10# icinga::DaemonCommand::Run(boost::program_options::variables_map const&, std::vector<std::string, std::allocator<std::string> > const&) const in /usr/lib64/icinga2/sbin/icinga2
11# 0x0000000000909920 in /usr/lib64/icinga2/sbin/icinga2
12# main in /usr/lib64/icinga2/sbin/icinga2
13# __libc_start_main in /lib64/libc.so.6
14# 0x0000000000906341 in /usr/lib64/icinga2/sbin/icinga2
Context:
(0) Activating object 'api' of type 'ApiListener'
To Reproduce
Just happened, So far I have no idea why.
Expected behavior
Not to crash.
Your Environment
Include as many relevant details about the environment you experienced the problem in
- Version used (
icinga2 --version):
icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: 2.13.2-1)
Copyright (c) 2012-2022 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <https://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
System information:
Platform: CentOS Linux
Platform version: 7 (Core)
Kernel: Linux
Kernel version: 3.10.0-1160.42.2.el7.x86_64
Architecture: x86_64
Build information:
Compiler: GNU 4.8.5
Build host: runner-hh8q3bz2-project-322-concurrent-0
OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017
Application information:
General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2
Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var
Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
- Operating System and version:
cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)
- Enabled features (
icinga2 feature list):
icinga2 feature list
Disabled features: command compatlog debuglog elasticsearch gelf graphite icingadb influxdb influxdb2 livestatus notification opentsdb perfdata statusdata syslog
Enabled features: api checker mainlog
- Config validation (
icinga2 daemon -C):
icinga2 daemon -C
[2022-02-21 08:19:28 +0100] information/cli: Icinga application loader (version: 2.13.2-1)
[2022-02-21 08:19:28 +0100] information/cli: Loading configuration file(s).
[2022-02-21 08:19:28 +0100] information/ConfigItem: Committing config item(s).
[2022-02-21 08:19:28 +0100] information/ApiListener: My API identity: enter-infi-test.alig.hu
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 1 CheckerComponent.
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 4 Zones.
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 1 IcingaApplication.
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 2 Endpoints.
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 1 FileLogger.
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 245 CheckCommands.
[2022-02-21 08:19:28 +0100] information/ConfigItem: Instantiated 1 ApiListener.
[2022-02-21 08:19:28 +0100] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2022-02-21 08:19:28 +0100] information/cli: Finished validating the configuration file(s).
Can you reproduce this behavior?
There must have been some error while writing /var/cache/icinga2/api-state.json (or a temporary file with some random characters appended). So please check if the /var/cache/icinga2/ directory is properly writable for the icinga user and the filesystem is not out of space or something like that.
If the problem persists, you can try to run icinga using strace (strace -f -yy icinga2 daemon) and look for lines related to the file mentioned above.
