wazuh icon indicating copy to clipboard operation
wazuh copied to clipboard

Added logic to avoid feed update during file processing

Open MiguelazoDS opened this issue 6 months ago • 1 comments

Related issue
#24967

Description

  • An undesired loop behavior was reported during the feed update.

  • After some research, the issue resulted from the lack of communication between the file processing in the database feed manager and the scheduled action managed by an interval time value.

Brief explanation

What it was found is that once the content manager finishes downloading all the offset files to be processed, the file processing starts, but this mechanism can last longer than the interval value set (60m minimum), so when the content manager triggers the action again, the processing is still in course which causes that all on-demand actions to update the offset value are ignored since the action initiated by the interval came first, and during the file processing the files are being replaced by the downloading task, which corrupts the processing.

Logs/Alerts example

First implementation

Expand

Offsets files were downloaded only once image

The on-demand events is the same as the files to be processed, and none of them were ignored image

The scheduled action only occurred twice. The first one at start, and the second one after the file processing ended. image

After the offset was updated the new scheduled action started immediately image

Update 08/16

ossec.log

Downloading image

On-demand events (update offset) image

Scheduled starts immediately after the processing has finished. image

Postponed events

2024/08/16 12:39:52 wazuh-modulesd:content-updater[160329] action.hpp:134 at operator()(): DEBUG: File processing in progress, scheduled request postponed.
2024/08/16 12:40:52 wazuh-modulesd:content-updater[160329] action.hpp:134 at operator()(): DEBUG: File processing in progress, scheduled request postponed.

Starting scheduled events

2024/08/16 12:38:39 wazuh-modulesd:content-updater[160329] action.hpp:151 at runActionScheduled(): DEBUG: Starting scheduled action for 'vulnerability_feed_manager'
2024/08/16 12:41:28 wazuh-modulesd:content-updater[160329] action.hpp:151 at runActionScheduled(): DEBUG: Starting scheduled action for 'vulnerability_feed_manager'
2024/08/16 12:42:29 wazuh-modulesd:content-updater[160329] action.hpp:151 at runActionScheduled(): DEBUG: Starting scheduled action for 'vulnerability_feed_manager'

Tests

Efficacy tests are failing but not related to this implementation. Some fixes and skips were added

False positives cases were disabled. 2024-08-07_17-23 2024-08-07_17-23_1

MiguelazoDS avatar Aug 07 '24 15:08 MiguelazoDS