mcholste
No new data in database
This is a strange issue and I have been working on it for over a week and cannot figure out the issue. This is a new build Ubuntu 14.04. The install.sh file went through without an issue but for some reason I am not getting any new data in the database. I received 100 logs and that is all. If I reboot it I receive 100 more additional logs. If I manually execute syslog-ng -Fevd it shows a multitude of data on screen. I see no issues in any of the log files. If I log into mysql and run select * from tables; I see that the start and end times of the syslog_data.syslogs_index_1 table are 10 seconds apart.
There is 1 exception I have found. If i execute livetail.pl I see everything that elsa is doing and all of that data is put into the database but only searchable from the archive. The moment I end the livetail the logs stop showing up in the database. I cannot figure where the disconnect is. Please assist in troubleshooting. Thank you.
The same problem here.
Fixed by commenting in fags(fow-control) and restating syslog-ng:
log { source(s_network); source(s_realtime); rewrite(r_host); rewrite(r_cisco_program); rewrite(r_snare); rewrite(r_from_pipes); rewrite(r_pipes); parser(p_db); rewrite(r_extracted_host); ###FILTER_UNPARSED###log { filter(f_unclassified); rewrite(r_unparsed); destination(d_unclassified); flags(final); }; log { destination(d_elsa); }; log { destination(d_debug); }; #flags(flow-control); };
Actually after few days tweaking syslog-ng, I see DB is indexing data but returns no results, only error: `query: SELECT CONCAT(SUBSTR(type, 1, 4), "_", id) AS name, start AS start_int, FROM_UNIXTIME(start) AS start, end AS end_int, FROM_UNIXTIME(end) AS end, type, last_id-first_id AS records, index_schema FROM syslog.indexes WHERE type="temporary" OR (type="permanent" AND ISNULL(locked_by)) OR type="realtime" ORDER BY start values:
- ERROR [2017/03/10 10:37:34] /usr/local/elsa/web/lib/SyncMysql.pm (64) SyncMysql::query 26472 [undef] Query: SELECT CONCAT(SUBSTR(type, 1, 4), "_", id) AS name, start AS start_int, FROM_UNIXTIME(start) AS start, end AS end_int, FROM_UNIXTIME(end) AS end, type, last_id-first_id AS records, index_schema FROM syslog.indexes WHERE type="temporary" OR (type="permanent" AND ISNULL(locked_by)) OR type="realtime" ORDER BY start with values got error JSON text must be an object or array (but found number, string, true, false or null, use allow_nonref to allow this) at /usr/local/elsa/web/lib/Utils.pm line 264.
- ERROR [2017/03/10 10:37:34] /usr/local/elsa/web/lib/Utils.pm (274) Utils::ANON 26472 [undef] No indexes, rv: 0 `
Before I disabled archive, at east searching from archive (archive:1) worked.
