duo_log_sync icon indicating copy to clipboard operation
duo_log_sync copied to clipboard

Published 20 hours ago • verified publisher icon duosecurity

adminaction consumer: failed to write some logs

Open jsandova opened this issue 4 years ago • 3 comments

I am getting the following error about adminaction unable to write to logs. Any ideas?

2020-12-22 23:19:50 INFO Starting DuoLogSync 2020-12-22 23:19:50 INFO DuoLogSync: Opening connection to ls01-dev-qa.aofk.net:2514 2020-12-22 23:19:50 INFO duo_client Admin initialized for ikey: *******, host: api-**.duosecurity.com 2020-12-22 23:19:50 ERROR Could not read checkpoint file for adminaction logs, consuming logs from {log_offset} timestamp 2020-12-22 23:19:50 ERROR Could not read checkpoint file for auth logs, consuming logs from {log_offset} timestamp 2020-12-22 23:19:50 INFO adminaction producer: fetching next logs after 120 seconds 2020-12-22 23:19:50 INFO adminaction consumer: waiting for logs 2020-12-22 23:19:50 INFO auth producer: fetching next logs after 120 seconds 2020-12-22 23:19:50 INFO auth consumer: waiting for logs 2020-12-22 23:21:50 INFO adminaction producer: fetching logs 2020-12-22 23:21:50 INFO auth producer: fetching logs Traceback (most recent call last): 2020-12-22 23:21:50 INFO adminaction producer: adding 57 logs to the queue 2020-12-22 23:21:50 INFO adminaction producer: added 57 logs to the queue 2020-12-22 23:21:50 INFO adminaction producer: fetching next logs after 120 seconds 2020-12-22 23:21:50 INFO adminaction consumer: received 57 logs from producer 2020-12-22 23:21:50 INFO adminaction consumer: writing logs 2020-12-22 23:21:50 WARNING adminaction consumer: failed to write some logs File "/usr/local/lib/python3.6/dist-packages/duologsync-2.0.0-py3.6.egg/duologsync/consumer/consumer.py", line 66, in consume File "/usr/local/lib/python3.6/dist-packages/duologsync-2.0.0-py3.6.egg/duologsync/writer.py", line 97, in write File "/usr/lib/python3.6/asyncio/streams.py", line 329, in drain raise exc File "/usr/lib/python3.6/asyncio/selector_events.py", line 714, in _read_ready data = self._sock.recv(self.max_size) ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/duologsync", line 11, in load_entry_point('duologsync==2.0.0', 'console_scripts', 'duologsync')() File "/usr/local/lib/python3.6/dist-packages/duologsync-2.0.0-py3.6.egg/duologsync/app.py", line 78, in main File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete return future.result() File "/usr/local/lib/python3.6/dist-packages/duologsync-2.0.0-py3.6.egg/duologsync/consumer/consumer.py", line 88, in consume File "/usr/local/lib/python3.6/dist-packages/duologsync-2.0.0-py3.6.egg/duologsync/producer/producer.py", line 205, in get_log_offset TypeError: 'NoneType' object is not subscriptable

jsandova avatar Dec 23 '20 07:12 jsandova

Here is my config.yml file.

version: '1.0.0' dls_settings: log_format: 'JSON' api: offset: 1 checkpointing: enabled: True directory: '/var/log/duo-logs' servers:

  • id: 'duo-logging' hostname: '10.176.18.45' port: 2514 protocol: 'TCP' account: ikey: '' skey: '' hostname: 'api-***.duosecurity.com' endpoint_server_mappings:
    • endpoints: ['adminaction', 'auth'] server: 'duo-logging' is_msp: False

jsandova avatar Dec 23 '20 07:12 jsandova

I was able to get it working by switching to UDP and using fluentd to forward the logs to our datadog logging console.

jsandova avatar Dec 24 '20 04:12 jsandova

I had a similar issue and also solved it by using UDP. Really annoying because we would like to use TCP :/

rka avatar Jan 04 '21 09:01 rka