wazuh-packages
wazuh-packages copied to clipboard
Published
20 hours ago •
wazuh
wazuh
Fix Wazuh indexer keystore unhandled message
| Wazuh version | Install type | Action performed | Platform |
|---|---|---|---|
| 4.3.5 | Wazuh indexer | All | All |
Hello team,
When trying to use the Wazuh indexer Keystore, the following message appears line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory. You can see here the full message:
/usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
The opensearch keystore does not exist. Do you want to create it? [y/N]y
Enter value for wazuh:
However, it seems the keystore is working as expected:
/usr/share/wazuh-indexer/bin/opensearch-keystore list
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
keystore.seed
wazuh
It seems the opensearch-keystore does not handle properly the config location if the installation path is different from the default one. We should fix this to avoid misleading the users.
Regards,
Miguel Casares
Currently we will need to include the OPENSEARCH_PATH_CONF variable to select where is keystore file stored. Example:
# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer ./opensearch-keystore add test
Enter value for test:
We could apply a workaround to fix the problem but it will remains in another files. We suggest to perform a fork of OpenSearch code and directly solve the main problem.
By the way, we should include a note about this matter into our documentation, example https://documentation.wazuh.com/current/migration-guide/wazuh-dashboard.html
This workround worked. However after running this, Wazuh-indexer restart failed with error permission denied on /etc/wazuh-indexer/opensearch-keystore. I had to 664 permission on this file for Indexer to start. Please include this also to the fix.
Hello Team,
I found that changing line 81 in the /usr/share/wazuh-indexer/bin/opensearch-env file for this, solved the issue:
https://github.com/wazuh/wazuh-indexer/blob/7469bf6e2603dbdbb808b96f453b4349c5f1d87f/distribution/src/bin/opensearch-env#L81-L83
if [ -z "$OPENSEARCH_PATH_CONF" ]; then OPENSEARCH_PATH_CONF=/etc/wazuh-indexer; fi
I hope this helps
Investigation
:red_circle: Test on 4.3.10 (CentOS 8)
The error is replicated when the keystore script is called without specifying OPENSEARCH_PATH_CONF, but it doesn't appear when the workaround is applied:
[root@centos8 vagrant]# bash wazuh-install.sh -a
13/03/2023 11:47:28 INFO: Starting Wazuh installation assistant. Wazuh version: 4.3.10
13/03/2023 11:47:28 INFO: Verbose logging redirected to /var/log/wazuh-install.log
13/03/2023 11:47:34 INFO: Wazuh repository added.
13/03/2023 11:47:34 INFO: --- Configuration files ---
13/03/2023 11:47:34 INFO: Generating configuration files.
13/03/2023 11:47:35 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
13/03/2023 11:47:35 INFO: --- Wazuh indexer ---
13/03/2023 11:47:35 INFO: Starting Wazuh indexer installation.
13/03/2023 11:50:46 INFO: Wazuh indexer installation finished.
13/03/2023 11:50:46 INFO: Wazuh indexer post-install configuration finished.
13/03/2023 11:50:46 INFO: Starting service wazuh-indexer.
13/03/2023 11:51:13 INFO: wazuh-indexer service started.
13/03/2023 11:51:13 INFO: Initializing Wazuh indexer cluster security settings.
13/03/2023 11:51:27 INFO: Wazuh indexer cluster initialized.
13/03/2023 11:51:27 INFO: --- Wazuh server ---
13/03/2023 11:51:27 INFO: Starting the Wazuh manager installation.
13/03/2023 11:54:06 INFO: Wazuh manager installation finished.
13/03/2023 11:54:06 INFO: Starting service wazuh-manager.
13/03/2023 11:54:29 INFO: wazuh-manager service started.
13/03/2023 11:54:29 INFO: Starting Filebeat installation.
13/03/2023 11:54:49 INFO: Filebeat installation finished.
13/03/2023 11:54:50 INFO: Filebeat post-install configuration finished.
13/03/2023 11:54:50 INFO: Starting service filebeat.
13/03/2023 11:54:51 INFO: filebeat service started.
13/03/2023 11:54:51 INFO: --- Wazuh dashboard ---
13/03/2023 11:54:51 INFO: Starting Wazuh dashboard installation.
13/03/2023 11:57:31 INFO: Wazuh dashboard installation finished.
13/03/2023 11:57:31 INFO: Wazuh dashboard post-install configuration finished.
13/03/2023 11:57:31 INFO: Starting service wazuh-dashboard.
13/03/2023 11:57:32 INFO: wazuh-dashboard service started.
13/03/2023 11:58:33 INFO: Initializing Wazuh dashboard web application.
13/03/2023 11:58:35 INFO: Wazuh dashboard web application initialized.
13/03/2023 11:58:35 INFO: --- Summary ---
13/03/2023 11:58:35 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: Et.jS*.6i+m2pN6QjmBgVv7pLN?MT9y5
13/03/2023 11:58:35 INFO: Installation finished.
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
The opensearch keystore does not exist. Do you want to create it? [y/N]y
Enter value for wazuh:
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
keystore.seed
wazuh
[root@centos8 vagrant]# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh2
Enter value for wazuh2:
[root@centos8 vagrant]# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh2
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch
opensearch.keystore opensearch-observability/ opensearch-reports-scheduler/ opensearch.yml
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 215 Mar 13 12:03 /etc/wazuh-indexer/opensearch.keystore
After a restart, the same error seen here happens, where the Wazuh Indexer won't start with an error, saying there are incorrect permissions on /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2023-03-13 12:07:53 UTC; 2min 18s ago
Docs: https://documentation.wazuh.com
Process: 745 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 745 (code=exited, status=1/FAILURE)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
Mar 13 12:07:53 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 12:07:53 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
Mar 13 12:07:53 centos8 systemd[1]: Failed to start Wazuh-indexer.
[root@centos8 vagrant]# tail /var/log/wazuh-indexer/
gc.log gc.log.02 wazuh-cluster_index_indexing_slowlog.json wazuh-cluster_index_search_slowlog.log
gc.log.00 wazuh-cluster_deprecation.json wazuh-cluster_index_indexing_slowlog.log wazuh-cluster.log
gc.log.01 wazuh-cluster_deprecation.log wazuh-cluster_index_search_slowlog.json wazuh-cluster_server.json
[root@centos8 vagrant]# tail /var/log/wazuh-indexer/wazuh-cluster.log
[2023-03-13T12:01:13,114][INFO ][o.o.j.s.JobSweeper ] [node-1] Running full sweep
[2023-03-13T12:05:06,000][INFO ][o.o.c.m.MetadataMappingService] [node-1] [wazuh-alerts-4.x-2023.03.13/FLtAwPWmTEiB979H2Xgj9A] update_mapping [_doc]
[2023-03-13T12:06:13,115][INFO ][o.o.j.s.JobSweeper ] [node-1] Running full sweep
[2023-03-13T12:06:46,320][INFO ][o.o.s.a.r.AuditMessageRouter] [node-1] Closing AuditMessageRouter
[2023-03-13T12:06:46,348][INFO ][o.o.s.a.s.SinkProvider ] [node-1] Closing DebugSink
[2023-03-13T12:06:46,324][INFO ][o.o.n.Node ] [node-1] stopping ...
[2023-03-13T12:06:48,939][INFO ][o.o.n.Node ] [node-1] stopped
[2023-03-13T12:06:48,940][INFO ][o.o.n.Node ] [node-1] closing ...
[2023-03-13T12:06:48,960][INFO ][o.o.s.a.i.AuditLogImpl ] [node-1] Closing AuditLogImpl
[2023-03-13T12:06:48,969][INFO ][o.o.n.Node ] [node-1] closed
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:83)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:241)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:225)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:56)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:07:50 centos8 systemd-entrypoint[745]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:83)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:241)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:225)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
Mar 13 12:07:53 centos8 systemd-entrypoint[745]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
Mar 13 12:07:53 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 12:07:53 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'exit-code'.
Mar 13 12:07:53 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because the control process exited with error code.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 215 Mar 13 12:03 /etc/wazuh-indexer/opensearch.keystore
When the permissions are changed and the Wazuh indexer is started again, a different error appears, this time saying that there is an unknown secure setting has been detected. This secure setting is the item previously added to the keystore using the workaround, which would led us to believe that the keystore used is the one created when the executable was called without the workaround.
It has been discovered a new keystore file is created the first time the executable is called without the workaround.
After removing this file, the error with the secure setting still appears, and after removing that item from the correct keystore using the executable, the file goes back to its original permissions
Only after changing its permissions back to 664 the Wazuh indexer can be restarted.
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 215 Mar 13 12:03 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl start wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 12:15:32 centos8 systemd[1]: Starting Wazuh-indexer...
-- Subject: Unit wazuh-indexer.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has begun starting up.
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: uncaught exception in thread [main]
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: java.lang.IllegalArgumentException: unknown secure setting [wazuh2] please check that any required plugins are installed, or check the breaking changes document>
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.node.Node.<init>(Node.java:463)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.node.Node.<init>(Node.java:319)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
Mar 13 12:15:53 centos8 systemd-entrypoint[4432]: For complete error details, refer to the log at /var/log/wazuh-indexer/wazuh-cluster.log
Mar 13 12:16:47 centos8 systemd[1]: wazuh-indexer.service: start operation timed out. Terminating.
Mar 13 12:16:47 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'timeout'.
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
keystore.seed
wazuh
[root@centos8 vagrant]#
[root@centos8 vagrant]# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh2
[root@centos8 vagrant]# find / | grep opensearch.keystore
/etc/wazuh-indexer/opensearch.keystore
/etc/wazuh-indexer/.opensearch.keystore.initial_md5sum
/usr/share/wazuh-indexer/bin/opensearch-keystore
/home/vagrant/opensearch.keystore
[root@centos8 vagrant]# ls
opensearch.keystore wazuh-install-files.tar wazuh-install.sh
[root@centos8 vagrant]# rm opensearch.keystore
rm: remove regular file 'opensearch.keystore'? y
[root@centos8 vagrant]# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh2
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
ERROR: OpenSearch keystore not found at [/home/vagrant/opensearch.keystore]. Use 'create' command to create one.
[root@centos8 vagrant]# ls
wazuh-install-files.tar wazuh-install.sh
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 12:31:24 centos8 systemd[1]: Starting Wazuh-indexer...
-- Subject: Unit wazuh-indexer.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has begun starting up.
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: uncaught exception in thread [main]
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: java.lang.IllegalArgumentException: unknown secure setting [wazuh2] please check that any required plugins are installed, or check the breaking changes document>
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:500)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:470)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:161)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.node.Node.<init>(Node.java:463)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.node.Node.<init>(Node.java:319)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
Mar 13 12:31:35 centos8 systemd-entrypoint[4952]: For complete error details, refer to the log at /var/log/wazuh-indexer/wazuh-cluster.log
Mar 13 12:32:39 centos8 systemd[1]: wazuh-indexer.service: start operation timed out. Terminating.
Mar 13 12:32:39 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'timeout'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'timeout'.
Mar 13 12:32:39 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
/usr/share/wazuh-indexer/bin/opensearch-env: line 89: cd: /usr/share/wazuh-indexer/config: No such file or directory
ERROR: OpenSearch keystore not found at [/home/vagrant/opensearch.keystore]. Use 'create' command to create one.
[root@centos8 vagrant]# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh2
[root@centos8 vagrant]# OPENSEARCH_PATH_CONF=/etc/wazuh-indexer /usr/share/wazuh-indexer/bin/opensearch-keystore remove wazuh2
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because the control process exited with error code.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:83)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:241)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:225)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:56)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:34:20 centos8 systemd-entrypoint[5727]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:83)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:241)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:225)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
Mar 13 12:34:21 centos8 systemd-entrypoint[5727]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
Mar 13 12:34:21 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 12:34:21 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'exit-code'.
Mar 13 12:34:21 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 196 Mar 13 12:34 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 196 Mar 13 12:34 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2023-03-13 12:35:33 UTC; 8s ago
Docs: https://documentation.wazuh.com
Main PID: 5951 (java)
Tasks: 58 (limit: 24912)
Memory: 2.2G
CGroup: /system.slice/wazuh-indexer.service
└─5951 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=tru>
Mar 13 12:35:14 centos8 systemd[1]: Starting Wazuh-indexer...
Mar 13 12:35:30 centos8 systemd-entrypoint[5951]: WARNING: An illegal reflective access operation has occurred
Mar 13 12:35:30 centos8 systemd-entrypoint[5951]: WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/wazuh-indexer/plugins/opensearch-anomaly-detection/proto>
Mar 13 12:35:30 centos8 systemd-entrypoint[5951]: WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
Mar 13 12:35:30 centos8 systemd-entrypoint[5951]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Mar 13 12:35:30 centos8 systemd-entrypoint[5951]: WARNING: All illegal access operations will be denied in a future release
Mar 13 12:35:33 centos8 systemd[1]: Started Wazuh-indexer.
[root@centos8 vagrant]#
:orange_circle: Test on 4.4.0 (CentOS 7)
Using 4.4.0 on CentOS 7 the error has not been replicated. When executable /usr/share/wazuh-indexer/ is called without the workaround, no error is shown.
[vagrant@centos7 unattended_installer]$ rm wazuh-install.sh
rm: remove write-protected regular file ‘wazuh-install.sh’? y
[vagrant@centos7 unattended_installer]$ bash builder.sh -i -d
[vagrant@centos7 unattended_installer]$ sudo bash wazuh-install.sh -a
13/03/2023 12:07:09 INFO: Starting Wazuh installation assistant. Wazuh version: 4.4.0
13/03/2023 12:07:09 INFO: Verbose logging redirected to /var/log/wazuh-install.log
13/03/2023 12:07:28 INFO: Wazuh development repository added.
13/03/2023 12:07:28 INFO: --- Configuration files ---
13/03/2023 12:07:28 INFO: Generating configuration files.
13/03/2023 12:07:30 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
13/03/2023 12:07:30 INFO: --- Wazuh indexer ---
13/03/2023 12:07:30 INFO: Starting Wazuh indexer installation.
13/03/2023 12:11:48 INFO: Wazuh indexer installation finished.
13/03/2023 12:11:48 INFO: Wazuh indexer post-install configuration finished.
13/03/2023 12:11:48 INFO: Starting service wazuh-indexer.
13/03/2023 12:12:09 INFO: wazuh-indexer service started.
13/03/2023 12:12:10 INFO: Initializing Wazuh indexer cluster security settings.
13/03/2023 12:12:20 INFO: Wazuh indexer cluster initialized.
13/03/2023 12:12:20 INFO: --- Wazuh server ---
13/03/2023 12:12:20 INFO: Starting the Wazuh manager installation.
13/03/2023 12:14:13 INFO: Wazuh manager installation finished.
13/03/2023 12:14:14 INFO: Starting service wazuh-manager.
13/03/2023 12:14:32 INFO: wazuh-manager service started.
13/03/2023 12:14:32 INFO: Starting Filebeat installation.
13/03/2023 12:15:05 INFO: Filebeat installation finished.
13/03/2023 12:15:07 INFO: Filebeat post-install configuration finished.
13/03/2023 12:15:07 INFO: Starting service filebeat.
13/03/2023 12:15:07 INFO: filebeat service started.
13/03/2023 12:15:07 INFO: --- Wazuh dashboard ---
13/03/2023 12:15:07 INFO: Starting Wazuh dashboard installation.
13/03/2023 12:19:07 INFO: Wazuh dashboard installation finished.
13/03/2023 12:19:07 INFO: Wazuh dashboard post-install configuration finished.
13/03/2023 12:19:07 INFO: Starting service wazuh-dashboard.
13/03/2023 12:19:08 INFO: wazuh-dashboard service started.
13/03/2023 12:20:03 INFO: Initializing Wazuh dashboard web application.
13/03/2023 12:20:04 INFO: Wazuh dashboard web application initialized.
13/03/2023 12:20:04 INFO: --- Summary ---
13/03/2023 12:20:04 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: .?UXXEF3T93s3ziRvY0+OxAsgLGMrk4o
13/03/2023 12:20:04 INFO: Installation finished.
[vagrant@centos7 unattended_installer]$ sudo su
[root@centos7 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh
Enter value for wazuh:
[root@centos7 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh
After adding a new item to the keystore and restarting the machine, the problems do appear. After restarting, the Wazuh indexer is down because of the permissions of /etc/wazuh-indexer/opensearch.keystore. When they are fixed and a restart of the service is tried again, a new error appears saying the newly created item is an unknown secure setting. If it is removed, the permissions are changed again. After correcting them, the Wazuh indexer is finally able to start.
[root@centos7 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2023-03-13 12:46:32 UTC; 20min ago
Docs: https://documentation.wazuh.com
Process: 718 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 718 (code=exited, status=1/FAILURE)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 12:46:32 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 12:46:32 centos7 systemd[1]: Failed to start Wazuh-indexer.
Mar 13 12:46:32 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 12:46:32 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:46:29 centos7 systemd-entrypoint[718]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 12:46:31 centos7 systemd-entrypoint[718]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 12:46:31 centos7 systemd-entrypoint[718]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 12:46:31 centos7 systemd-entrypoint[718]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 12:46:31 centos7 systemd-entrypoint[718]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 12:46:32 centos7 systemd-entrypoint[718]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 12:46:32 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 12:46:32 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 12:46:32 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 12:46:32 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 212 Mar 13 12:43 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 212 Mar 13 12:43 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded. See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
-- The result is failed.
Mar 13 12:46:32 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 12:46:32 centos7 systemd[1]: wazuh-indexer.service failed.
Mar 13 13:07:47 centos7 systemd[1]: Starting Wazuh-indexer...
-- Subject: Unit wazuh-indexer.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has begun starting up.
Mar 13 13:07:50 centos7 systemd-entrypoint[3907]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 13:07:50 centos7 systemd-entrypoint[3907]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 13:07:50 centos7 systemd-entrypoint[3907]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 13:07:50 centos7 systemd-entrypoint[3907]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 13:07:53 centos7 systemd-entrypoint[3907]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 13:07:53 centos7 systemd-entrypoint[3907]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 13:07:53 centos7 systemd-entrypoint[3907]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 13:07:53 centos7 systemd-entrypoint[3907]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: uncaught exception in thread [main]
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: java.lang.IllegalArgumentException: unknown secure setting [wazuh] please check that any required plugins are installed, or check the breaking changes documentat
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:591)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:532)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:502)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:472)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:170)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.node.Node.<init>(Node.java:496)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.node.Node.<init>(Node.java:353)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 13:08:07 centos7 systemd-entrypoint[3907]: For complete error details, refer to the log at /var/log/wazuh-indexer/wazuh-cluster.log
Mar 13 13:10:47 centos7 systemd[1]: wazuh-indexer.service start operation timed out. Terminating.
Mar 13 13:10:48 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 13:10:48 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 13:10:48 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore remove wazuh
[root@centos7 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because the control process exited with error code. See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 13:16:24 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 13:16:24 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 13:16:24 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 13:16:24 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 13:16:22 centos7 systemd-entrypoint[4209]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 13:16:23 centos7 systemd-entrypoint[4209]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 13:16:23 centos7 systemd-entrypoint[4209]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 13:16:23 centos7 systemd-entrypoint[4209]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 13:16:23 centos7 systemd-entrypoint[4209]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 13:16:24 centos7 systemd-entrypoint[4209]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 13:16:24 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 13:16:24 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 13:16:24 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 13:16:24 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 196 Mar 13 13:16 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 196 Mar 13 13:16 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# systemctl restart wazuh-indexer
[root@centos7 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2023-03-13 13:17:26 UTC; 3min 38s ago
Docs: https://documentation.wazuh.com
Main PID: 4434 (java)
CGroup: /system.slice/wazuh-indexer.service
└─4434 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=t...
Mar 13 13:17:00 centos7 systemd[1]: Starting Wazuh-indexer...
Mar 13 13:17:03 centos7 systemd-entrypoint[4434]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 13:17:03 centos7 systemd-entrypoint[4434]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 13:17:03 centos7 systemd-entrypoint[4434]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 13:17:03 centos7 systemd-entrypoint[4434]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 13:17:07 centos7 systemd-entrypoint[4434]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 13:17:07 centos7 systemd-entrypoint[4434]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 13:17:07 centos7 systemd-entrypoint[4434]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 13:17:07 centos7 systemd-entrypoint[4434]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 13:17:26 centos7 systemd[1]: Started Wazuh-indexer.
[root@centos7 vagrant]#
:orange_circle: Test on 4.4.0 (CentOS 8)
Tried on CentOS 8 with exactly the same output as with CentOS 7
[root@centos8 unattended_installer]# bash wazuh-install.sh -a
13/03/2023 13:06:24 INFO: Starting Wazuh installation assistant. Wazuh version: 4.4.0
13/03/2023 13:06:24 INFO: Verbose logging redirected to /var/log/wazuh-install.log
z13/03/2023 13:06:31 INFO: Wazuh development repository added.
13/03/2023 13:06:31 INFO: --- Configuration files ---
13/03/2023 13:06:31 INFO: Generating configuration files.
13/03/2023 13:06:32 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
13/03/2023 13:06:33 INFO: --- Wazuh indexer ---
13/03/2023 13:06:33 INFO: Starting Wazuh indexer installation.
13/03/2023 13:06:36 ERROR: Wazuh indexer installation failed.
13/03/2023 13:06:36 INFO: --- Removing existing Wazuh installation ---
13/03/2023 13:06:36 INFO: Installation cleaned. Check the /var/log/wazuh-install.log file to learn more about the issue.
[root@centos8 unattended_installer]# bash /vagrant/scripts/centos-sources.sh
[root@centos8 unattended_installer]# bash wazuh-install.sh -a
13/03/2023 13:30:13 INFO: Starting Wazuh installation assistant. Wazuh version: 4.4.0
13/03/2023 13:30:13 INFO: Verbose logging redirected to /var/log/wazuh-install.log
13/03/2023 13:30:20 INFO: Wazuh development repository added.
13/03/2023 13:30:20 INFO: --- Configuration files ---
13/03/2023 13:30:20 INFO: Generating configuration files.
13/03/2023 13:30:22 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
13/03/2023 13:30:22 INFO: --- Wazuh indexer ---
13/03/2023 13:30:22 INFO: Starting Wazuh indexer installation.
13/03/2023 13:36:59 INFO: Wazuh indexer installation finished.
13/03/2023 13:37:00 INFO: Wazuh indexer post-install configuration finished.
13/03/2023 13:37:00 INFO: Starting service wazuh-indexer.
13/03/2023 13:37:32 INFO: wazuh-indexer service started.
13/03/2023 13:37:32 INFO: Initializing Wazuh indexer cluster security settings.
13/03/2023 13:37:42 INFO: Wazuh indexer cluster initialized.
13/03/2023 13:37:42 INFO: --- Wazuh server ---
13/03/2023 13:37:42 INFO: Starting the Wazuh manager installation.
13/03/2023 13:41:44 INFO: Wazuh manager installation finished.
13/03/2023 13:41:44 INFO: Starting service wazuh-manager.
13/03/2023 13:42:02 INFO: wazuh-manager service started.
13/03/2023 13:42:02 INFO: Starting Filebeat installation.
13/03/2023 13:42:29 INFO: Filebeat installation finished.
13/03/2023 13:42:32 INFO: Filebeat post-install configuration finished.
13/03/2023 13:42:32 INFO: Starting service filebeat.
13/03/2023 13:42:33 INFO: filebeat service started.
13/03/2023 13:42:33 INFO: --- Wazuh dashboard ---
13/03/2023 13:42:33 INFO: Starting Wazuh dashboard installation.
13/03/2023 13:50:41 INFO: Wazuh dashboard installation finished.
13/03/2023 13:50:41 INFO: Wazuh dashboard post-install configuration finished.
13/03/2023 13:50:41 INFO: Starting service wazuh-dashboard.
13/03/2023 13:50:42 INFO: wazuh-dashboard service started.
13/03/2023 13:51:23 INFO: Initializing Wazuh dashboard web application.
13/03/2023 13:51:26 INFO: Wazuh dashboard web application initialized.
13/03/2023 13:51:26 INFO: --- Summary ---
13/03/2023 13:51:26 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: 7GKgi9g?KLGO6L*l3eFlUFdxN6oD8sTW
13/03/2023 13:51:26 INFO: Installation finished.
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh
Enter value for wazuh:
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore add list
Enter value for list: [root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh
After a machine reboot:
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2023-03-13 15:30:08 UTC; 46s ago
Docs: https://documentation.wazuh.com
Process: 747 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 747 (code=exited, status=1/FAILURE)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 15:30:08 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 15:30:08 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
Mar 13 15:30:08 centos8 systemd[1]: Failed to start Wazuh-indexer.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 15:30:05 centos8 systemd-entrypoint[747]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 15:30:07 centos8 systemd-entrypoint[747]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 15:30:07 centos8 systemd-entrypoint[747]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 15:30:07 centos8 systemd-entrypoint[747]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 15:30:07 centos8 systemd-entrypoint[747]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 15:30:08 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 15:30:08 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 15:30:08 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'exit-code'.
Mar 13 15:30:08 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 214 Mar 13 15:28 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 214 Mar 13 15:28 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
-- Subject: Unit wazuh-indexer.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has begun starting up.
Mar 13 15:32:11 centos8 systemd-entrypoint[4297]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 15:32:11 centos8 systemd-entrypoint[4297]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 15:32:11 centos8 systemd-entrypoint[4297]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 15:32:11 centos8 systemd-entrypoint[4297]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 15:32:15 centos8 systemd-entrypoint[4297]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 15:32:15 centos8 systemd-entrypoint[4297]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 15:32:15 centos8 systemd-entrypoint[4297]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 15:32:15 centos8 systemd-entrypoint[4297]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: uncaught exception in thread [main]
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: java.lang.IllegalArgumentException: unknown secure setting [wazuh] please check that any required plugins are installed, or check the breaking changes documenta>
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:591)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:532)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:502)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:472)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:170)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.node.Node.<init>(Node.java:496)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.node.Node.<init>(Node.java:353)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 15:32:37 centos8 systemd-entrypoint[4297]: For complete error details, refer to the log at /var/log/wazuh-indexer/wazuh-cluster.log
Mar 13 15:35:08 centos8 systemd[1]: wazuh-indexer.service: start operation timed out. Terminating.
Mar 13 15:35:08 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'timeout'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'timeout'.
Mar 13 15:35:08 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore remove wazuh
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because the control process exited with error code.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 15:36:20 centos8 systemd-entrypoint[4586]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 15:36:21 centos8 systemd-entrypoint[4586]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 15:36:21 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 15:36:21 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'exit-code'.
Mar 13 15:36:21 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 196 Mar 13 15:36 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 196 Mar 13 15:36 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2023-03-13 15:37:12 UTC; 8s ago
Docs: https://documentation.wazuh.com
Main PID: 4805 (java)
Tasks: 63 (limit: 24912)
Memory: 2.2G
CGroup: /system.slice/wazuh-indexer.service
└─4805 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=tru>
Mar 13 15:36:46 centos8 systemd[1]: Starting Wazuh-indexer...
Mar 13 15:36:49 centos8 systemd-entrypoint[4805]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 15:36:49 centos8 systemd-entrypoint[4805]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 15:36:49 centos8 systemd-entrypoint[4805]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 15:36:49 centos8 systemd-entrypoint[4805]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 15:36:51 centos8 systemd-entrypoint[4805]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 15:36:51 centos8 systemd-entrypoint[4805]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 15:36:51 centos8 systemd-entrypoint[4805]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 15:36:51 centos8 systemd-entrypoint[4805]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 15:37:12 centos8 systemd[1]: Started Wazuh-indexer.
[root@centos8 vagrant]#
:orange_circle: Test on 4.5.0 (CentOS 7 & CentOS 8)
Both for CentOS 7 and CentOS 8, when tried with version 4.5, the result is exactly the same than with version 4.4.0.
CentOS 7
[root@centos7 unattended_installer]# bash wazuh-install.sh -a
13/03/2023 15:42:42 INFO: Starting Wazuh installation assistant. Wazuh version: 4.5.0
13/03/2023 15:42:42 INFO: Verbose logging redirected to /var/log/wazuh-install.log
13/03/2023 15:42:55 INFO: Wazuh development repository added.
13/03/2023 15:42:55 INFO: --- Configuration files ---
13/03/2023 15:42:55 INFO: Generating configuration files.
13/03/2023 15:42:56 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
13/03/2023 15:42:57 INFO: --- Wazuh indexer ---
13/03/2023 15:42:57 INFO: Starting Wazuh indexer installation.
13/03/2023 15:46:35 INFO: Wazuh indexer installation finished.
13/03/2023 15:46:35 INFO: Wazuh indexer post-install configuration finished.
13/03/2023 15:46:35 INFO: Starting service wazuh-indexer.
13/03/2023 15:46:57 INFO: wazuh-indexer service started.
13/03/2023 15:46:57 INFO: Initializing Wazuh indexer cluster security settings.
13/03/2023 15:47:08 INFO: Wazuh indexer cluster initialized.
13/03/2023 15:47:08 INFO: --- Wazuh server ---
13/03/2023 15:47:08 INFO: Starting the Wazuh manager installation.
13/03/2023 15:49:09 INFO: Wazuh manager installation finished.
13/03/2023 15:49:09 INFO: Starting service wazuh-manager.
13/03/2023 15:49:26 INFO: wazuh-manager service started.
13/03/2023 15:49:26 INFO: Starting Filebeat installation.
13/03/2023 15:49:45 INFO: Filebeat installation finished.
13/03/2023 15:49:47 INFO: Filebeat post-install configuration finished.
13/03/2023 15:49:47 INFO: Starting service filebeat.
13/03/2023 15:49:47 INFO: filebeat service started.
13/03/2023 15:49:47 INFO: --- Wazuh dashboard ---
13/03/2023 15:49:47 INFO: Starting Wazuh dashboard installation.
13/03/2023 15:57:15 INFO: Wazuh dashboard installation finished.
13/03/2023 15:57:15 INFO: Wazuh dashboard post-install configuration finished.
13/03/2023 15:57:15 INFO: Starting service wazuh-dashboard.
13/03/2023 15:57:16 INFO: wazuh-dashboard service started.
13/03/2023 15:58:14 INFO: Initializing Wazuh dashboard web application.
13/03/2023 15:58:15 INFO: Wazuh dashboard web application initialized.
13/03/2023 15:58:15 INFO: --- Summary ---
13/03/2023 15:58:15 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: EaVSS72nk6MriGHsXUFE.s9p5J+*27IF
13/03/2023 15:58:16 INFO: Installation finished.
[root@centos7 unattended_installer]# /usr/share/wazuh-indexer/
bin/ .cache/ jdk/ lib/ modules/ performance-analyzer-rca/ plugins/
[root@centos7 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh
Enter value for wazuh:
[root@centos7 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh
After reboot:
[root@centos7 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2023-03-13 16:07:35 UTC; 1min 59s ago
Docs: https://documentation.wazuh.com
Process: 717 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 717 (code=exited, status=1/FAILURE)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 16:07:35 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 16:07:35 centos7 systemd[1]: Failed to start Wazuh-indexer.
Mar 13 16:07:35 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 16:07:35 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# journalctl -xeu
journalctl: option requires an argument -- 'u'
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:07:32 centos7 systemd-entrypoint[717]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 16:07:34 centos7 systemd-entrypoint[717]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:07:34 centos7 systemd-entrypoint[717]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:07:34 centos7 systemd-entrypoint[717]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:07:34 centos7 systemd-entrypoint[717]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 16:07:35 centos7 systemd-entrypoint[717]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 16:07:35 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 16:07:35 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 16:07:35 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 16:07:35 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 214 Mar 13 16:05 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 214 Mar 13 16:05 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded. See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos7 vagrant]# ^C
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
-- The result is failed.
Mar 13 16:07:35 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 16:07:35 centos7 systemd[1]: wazuh-indexer.service failed.
Mar 13 16:11:38 centos7 systemd[1]: Starting Wazuh-indexer...
-- Subject: Unit wazuh-indexer.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has begun starting up.
Mar 13 16:11:42 centos7 systemd-entrypoint[3838]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:11:42 centos7 systemd-entrypoint[3838]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:11:42 centos7 systemd-entrypoint[3838]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:11:42 centos7 systemd-entrypoint[3838]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:11:51 centos7 systemd-entrypoint[3838]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:11:51 centos7 systemd-entrypoint[3838]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:11:51 centos7 systemd-entrypoint[3838]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 16:11:51 centos7 systemd-entrypoint[3838]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: uncaught exception in thread [main]
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: java.lang.IllegalArgumentException: unknown secure setting [wazuh] please check that any required plugins are installed, or check the breaking changes documentat
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:591)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:532)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:502)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:472)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:170)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.node.Node.<init>(Node.java:496)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.node.Node.<init>(Node.java:353)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 16:13:13 centos7 systemd-entrypoint[3838]: For complete error details, refer to the log at /var/log/wazuh-indexer/wazuh-cluster.log
Mar 13 16:14:38 centos7 systemd[1]: wazuh-indexer.service start operation timed out. Terminating.
Mar 13 16:14:38 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 16:14:38 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 16:14:38 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore remove wazuh
[root@centos7 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because the control process exited with error code. See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos7 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:15:13 centos7 systemd-entrypoint[4138]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 16:15:14 centos7 systemd-entrypoint[4138]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:15:14 centos7 systemd-entrypoint[4138]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:15:14 centos7 systemd-entrypoint[4138]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:15:14 centos7 systemd-entrypoint[4138]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:15:15 centos7 systemd-entrypoint[4138]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:15:15 centos7 systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Mar 13 16:15:15 centos7 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
Mar 13 16:15:15 centos7 systemd[1]: Unit wazuh-indexer.service entered failed state.
Mar 13 16:15:15 centos7 systemd[1]: wazuh-indexer.service failed.
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 196 Mar 13 16:15 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 196 Mar 13 16:15 /etc/wazuh-indexer/opensearch.keystore
[root@centos7 vagrant]# systemctl restart wazuh-indexer
[root@centos7 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2023-03-13 16:16:58 UTC; 2min 47s ago
Docs: https://documentation.wazuh.com
Main PID: 4350 (java)
CGroup: /system.slice/wazuh-indexer.service
└─4350 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=t...
Mar 13 16:16:33 centos7 systemd[1]: Starting Wazuh-indexer...
Mar 13 16:16:35 centos7 systemd-entrypoint[4350]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:16:35 centos7 systemd-entrypoint[4350]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:16:35 centos7 systemd-entrypoint[4350]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:16:35 centos7 systemd-entrypoint[4350]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:16:37 centos7 systemd-entrypoint[4350]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:16:37 centos7 systemd-entrypoint[4350]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:16:37 centos7 systemd-entrypoint[4350]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 16:16:37 centos7 systemd-entrypoint[4350]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:16:58 centos7 systemd[1]: Started Wazuh-indexer.
CentOS 8
[root@centos8 unattended_installer]# bash wazuh-install.sh -a
13/03/2023 15:52:26 INFO: Starting Wazuh installation assistant. Wazuh version: 4.5.0
13/03/2023 15:52:26 INFO: Verbose logging redirected to /var/log/wazuh-install.log
13/03/2023 15:52:31 INFO: --- Dependencies ---
13/03/2023 15:52:31 INFO: Installing lsof.
13/03/2023 15:54:25 INFO: Wazuh development repository added.
13/03/2023 15:54:25 INFO: --- Configuration files ---
13/03/2023 15:54:25 INFO: Generating configuration files.
13/03/2023 15:54:26 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
13/03/2023 15:54:26 INFO: --- Wazuh indexer ---
13/03/2023 15:54:26 INFO: Starting Wazuh indexer installation.
13/03/2023 16:05:59 INFO: Wazuh indexer installation finished.
13/03/2023 16:05:59 INFO: Wazuh indexer post-install configuration finished.
13/03/2023 16:05:59 INFO: Starting service wazuh-indexer.
13/03/2023 16:06:41 INFO: wazuh-indexer service started.
13/03/2023 16:06:41 INFO: Initializing Wazuh indexer cluster security settings.
13/03/2023 16:06:52 INFO: Wazuh indexer cluster initialized.
13/03/2023 16:06:52 INFO: --- Wazuh server ---
13/03/2023 16:06:52 INFO: Starting the Wazuh manager installation.
13/03/2023 16:13:15 INFO: Wazuh manager installation finished.
13/03/2023 16:13:15 INFO: Starting service wazuh-manager.
13/03/2023 16:13:33 INFO: wazuh-manager service started.
13/03/2023 16:13:33 INFO: Starting Filebeat installation.
13/03/2023 16:14:11 INFO: Filebeat installation finished.
13/03/2023 16:14:13 INFO: Filebeat post-install configuration finished.
13/03/2023 16:14:13 INFO: Starting service filebeat.
13/03/2023 16:14:14 INFO: filebeat service started.
13/03/2023 16:14:14 INFO: --- Wazuh dashboard ---
13/03/2023 16:14:14 INFO: Starting Wazuh dashboard installation.
13/03/2023 16:23:39 INFO: Wazuh dashboard installation finished.
13/03/2023 16:23:39 INFO: Wazuh dashboard post-install configuration finished.
13/03/2023 16:23:39 INFO: Starting service wazuh-dashboard.
13/03/2023 16:23:39 INFO: wazuh-dashboard service started.
13/03/2023 16:24:41 INFO: Initializing Wazuh dashboard web application.
13/03/2023 16:24:52 INFO: Wazuh dashboard web application initialized.
13/03/2023 16:24:52 INFO: --- Summary ---
13/03/2023 16:24:52 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: WTjEzpZCPGr.dQM?IhLvOMXwc930lTAH
13/03/2023 16:24:52 INFO: Installation finished.
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh
Enter value for wazuh:
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore list
keystore.seed
wazuh
After reboot:
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2023-03-13 16:44:17 UTC; 5min ago
Docs: https://documentation.wazuh.com
Process: 735 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 735 (code=exited, status=1/FAILURE)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.main(Command.java:101)
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Mar 13 16:44:13 centos8 systemd-entrypoint[735]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:44:13 centos8 systemd-entrypoint[735]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:44:13 centos8 systemd-entrypoint[735]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:44:13 centos8 systemd-entrypoint[735]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:246)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:230)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 16:44:17 centos8 systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
Mar 13 16:44:17 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'exit-code'.
Mar 13 16:44:17 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 214 Mar 13 16:41 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 214 Mar 13 16:41 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer
-- Subject: Unit wazuh-indexer.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has begun starting up.
Mar 13 16:50:34 centos8 systemd-entrypoint[4250]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:50:34 centos8 systemd-entrypoint[4250]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:50:34 centos8 systemd-entrypoint[4250]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:50:34 centos8 systemd-entrypoint[4250]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:50:37 centos8 systemd-entrypoint[4250]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:50:37 centos8 systemd-entrypoint[4250]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:50:37 centos8 systemd-entrypoint[4250]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 16:50:37 centos8 systemd-entrypoint[4250]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: uncaught exception in thread [main]
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: java.lang.IllegalArgumentException: unknown secure setting [wazuh] please check that any required plugins are installed, or check the breaking changes documenta>
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:591)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:532)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:502)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:472)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.common.settings.SettingsModule.<init>(SettingsModule.java:170)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.node.Node.<init>(Node.java:496)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.node.Node.<init>(Node.java:353)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.cli.Command.main(Command.java:101)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: For complete error details, refer to the log at /var/log/wazuh-indexer/wazuh-cluster.log
Mar 13 16:53:31 centos8 systemd[1]: wazuh-indexer.service: start operation timed out. Terminating.
Mar 13 16:53:31 centos8 systemd[1]: wazuh-indexer.service: Failed with result 'timeout'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit wazuh-indexer.service has entered the 'failed' state with result 'timeout'.
Mar 13 16:53:31 centos8 systemd[1]: Failed to start Wazuh-indexer.
-- Subject: Unit wazuh-indexer.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit wazuh-indexer.service has failed.
--
-- The result is failed.
[root@centos8 vagrant]# /usr/share/wazuh-indexer/bin/opensearch-keystore remove wazuh
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because the control process exited with error code.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer | grep Exception
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:44:10 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:44:17 centos8 systemd-entrypoint[735]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: java.lang.IllegalArgumentException: unknown secure setting [wazuh] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
Mar 13 16:54:28 centos8 systemd-entrypoint[4551]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:54:28 centos8 systemd-entrypoint[4551]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:54:28 centos8 systemd-entrypoint[4551]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:54:28 centos8 systemd-entrypoint[4551]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 13 16:54:30 centos8 systemd-entrypoint[4551]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:54:30 centos8 systemd-entrypoint[4551]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 13 16:54:30 centos8 systemd-entrypoint[4551]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 13 16:54:30 centos8 systemd-entrypoint[4551]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 13 16:54:30 centos8 systemd-entrypoint[4551]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw----. 1 root root 196 Mar 13 16:54 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# ls -lah /etc/wazuh-indexer/opensearch.keystore
-rw-rw-r--. 1 root root 196 Mar 13 16:54 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2023-03-13 16:55:22 UTC; 36s ago
Docs: https://documentation.wazuh.com
Main PID: 4762 (java)
Tasks: 71 (limit: 24912)
Memory: 2.2G
CGroup: /system.slice/wazuh-indexer.service
└─4762 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=tru>
Mar 13 16:54:59 centos8 systemd[1]: Starting Wazuh-indexer...
Mar 13 16:55:01 centos8 systemd-entrypoint[4762]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:55:01 centos8 systemd-entrypoint[4762]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:55:01 centos8 systemd-entrypoint[4762]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Mar 13 16:55:01 centos8 systemd-entrypoint[4762]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:55:03 centos8 systemd-entrypoint[4762]: WARNING: A terminally deprecated method in java.lang.System has been called
Mar 13 16:55:03 centos8 systemd-entrypoint[4762]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.4.1.jar)
Mar 13 16:55:03 centos8 systemd-entrypoint[4762]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Mar 13 16:55:03 centos8 systemd-entrypoint[4762]: WARNING: System::setSecurityManager will be removed in a future release
Mar 13 16:55:22 centos8 systemd[1]: Started Wazuh-indexer.
[root@centos8 vagrant]#
Summary
After version 4.4.0, the first error for which this Issue was created, the unhandled error for /usr/share/wazuh-indexer/bin/opensearch-keystore, has been removed. The executable modifies the file it is supposed to modify, '/etc/wazuh-indexer/opensearch.keystore' and doesn't create any extra files. After modifying it, there are two errors.
- The permissions of the file after each modification are
660, and the indexer initialization fails if it can't read the file, so they must be664:
Mar 13 16:54:28 centos8 systemd-entrypoint[4551]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
- The second error is the new item added to the keystore is not correctly recognized. This second error can be probably solved by running the script to configure the security of the cluster again after the keystore modifications:
Mar 13 16:51:03 centos8 systemd-entrypoint[4250]: java.lang.IllegalArgumentException: unknown secure setting [wazuh] please check that any required plugins are installed, or check the breaking changes documenta>
Solutions
First tried solution: security initialization
A possible solution, especially for the error unknown secure setting, could have been re-initializing the indexer security, but it hasn't solved any of the two problems.
[root@centos8 unattended_installer]# bash wazuh-install.sh -a
14/03/2023 11:28:14 INFO: Starting Wazuh installation assistant. Wazuh version: 4.5.0
14/03/2023 11:28:14 INFO: Verbose logging redirected to /var/log/wazuh-install.log
14/03/2023 11:28:19 INFO: --- Dependencies ---
14/03/2023 11:28:19 INFO: Installing lsof.
14/03/2023 11:28:42 INFO: Wazuh development repository added.
14/03/2023 11:28:42 INFO: --- Configuration files ---
14/03/2023 11:28:42 INFO: Generating configuration files.
14/03/2023 11:28:44 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation.
14/03/2023 11:28:44 INFO: --- Wazuh indexer ---
14/03/2023 11:28:44 INFO: Starting Wazuh indexer installation.
14/03/2023 11:31:56 INFO: Wazuh indexer installation finished.
14/03/2023 11:31:56 INFO: Wazuh indexer post-install configuration finished.
14/03/2023 11:31:56 INFO: Starting service wazuh-indexer.
14/03/2023 11:32:15 INFO: wazuh-indexer service started.
14/03/2023 11:32:15 INFO: Initializing Wazuh indexer cluster security settings.
14/03/2023 11:32:25 INFO: Wazuh indexer cluster initialized.
14/03/2023 11:32:25 INFO: --- Wazuh server ---
14/03/2023 11:32:25 INFO: Starting the Wazuh manager installation.
14/03/2023 11:34:21 INFO: Wazuh manager installation finished.
14/03/2023 11:34:21 INFO: Starting service wazuh-manager.
14/03/2023 11:34:37 INFO: wazuh-manager service started.
14/03/2023 11:34:37 INFO: Starting Filebeat installation.
14/03/2023 11:34:50 INFO: Filebeat installation finished.
14/03/2023 11:34:52 INFO: Filebeat post-install configuration finished.
14/03/2023 11:34:52 INFO: Starting service filebeat.
14/03/2023 11:34:52 INFO: filebeat service started.
14/03/2023 11:34:52 INFO: --- Wazuh dashboard ---
14/03/2023 11:34:52 INFO: Starting Wazuh dashboard installation.
14/03/2023 11:37:35 INFO: Wazuh dashboard installation finished.
14/03/2023 11:37:36 INFO: Wazuh dashboard post-install configuration finished.
14/03/2023 11:37:36 INFO: Starting service wazuh-dashboard.
14/03/2023 11:37:36 INFO: wazuh-dashboard service started.
14/03/2023 11:38:08 INFO: Initializing Wazuh dashboard web application.
14/03/2023 11:38:10 INFO: Wazuh dashboard web application initialized.
14/03/2023 11:38:10 INFO: --- Summary ---
14/03/2023 11:38:10 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: .blxUdCUxL3q4nAK6lMYBtq+4Vm.q9Jv
14/03/2023 11:38:10 INFO: Installation finished.
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/
bin/ .cache/ jdk/ lib/ modules/ performance-analyzer-rca/ plugins/
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/opensearch-keystore add wazuh-test
Enter value for wazuh-test:
[root@centos8 unattended_installer]# /usr/share/wazuh-indexer/bin/indexer-security-init.sh
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
Security Admin v7
Will connect to 127.0.0.1:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.4.1
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml
SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml
SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml
SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml
SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml
SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml
SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml
SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml
SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml
SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml
SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success
After the reboot:
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2023-03-14 11:52:35 UTC; 1min 40s ago
Docs: https://documentation.wazuh.com
Process: 747 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 747 (code=exited, status=1/FAILURE)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer | grep Exception
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
[root@centos8 vagrant]# chmod 664 /etc/wazuh-indexer/opensearch.keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
Job for wazuh-indexer.service failed because a timeout was exceeded.
See "systemctl status wazuh-indexer.service" and "journalctl -xe" for details.
[root@centos8 vagrant]# journalctl -xeu wazuh-indexer | grep Exception
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 14 11:52:33 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Mar 14 11:52:35 centos8 systemd-entrypoint[747]: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Mar 14 11:55:07 centos8 systemd-entrypoint[4242]: java.lang.IllegalArgumentException: unknown secure setting [wazuh-test] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
[root@centos8 vagrant]#
Second solution
Another solution may be to somehow disable the use of the executable to the general user. Right now it is one more executable, not mentioned at all in the Wazuh documentation, and that breaks an installation if it is used, without a proper error or warning said. Furthermore, a test was done removing it and both restarting the Wazuh indexer and rebooting the machine, and no error has been seen:
[root@centos8 vagrant]# rm -f /usr/share/wazuh-indexer/bin/opensearch-keystore
[root@centos8 vagrant]# systemctl restart wazuh-indexer
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2023-03-14 13:33:03 UTC; 7s ago
Docs: https://documentation.wazuh.com
Main PID: 5105 (java)
Tasks: 61 (limit: 24912)
Memory: 2.2G
CGroup: /system.slice/wazuh-indexer.service
└─5105 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=tru>
After reboot:
[root@centos8 vagrant]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2023-03-14 13:42:39 UTC; 49min ago
Docs: https://documentation.wazuh.com
Main PID: 805 (java)
Tasks: 62 (limit: 24912)
Memory: 2.3G
CGroup: /system.slice/wazuh-indexer.service
└─805 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true>
[root@centos8 vagrant]# cat /var/log/wazuh-indexer/wazuh-cluster.log | grep error
[root@centos8 vagrant]# cat /var/log/wazuh-indexer/wazuh-cluster.log | grep warning
[root@centos8 vagrant]# cat /var/log/wazuh-indexer/gc.log | grep warning
[root@centos8 vagrant]# cat /var/log/wazuh-indexer/gc.log | grep error
[root@centos8 vagrant]#
This solution should be discussed with @snaow and probably the people responsible for the Wazuh indexer fork.
