beats icon indicating copy to clipboard operation
beats copied to clipboard

Elasticsearch output can report an incorrect count for active events

Open faec opened this issue 10 months ago • 5 comments

The metrics value monitoring.metrics.beat.libbeat.output.events.active can be skewed to inaccurate totals. I observed this in main after heavy ingestion that concluded (with all events ingested and no genuine active events) with an events.active of 959. filebeat.events.active showed the correct total of 0. (Of the two, Filebeat is usually the lagging metric, since it receives acknowledgment notifications later than the output.)

The run where this issue was observed involved both batch splitting and 529 errors from upstream, so either or both of those could be factors.

faec avatar Apr 22 '24 20:04 faec

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

elasticmachine avatar Apr 22 '24 20:04 elasticmachine

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

elasticmachine avatar Apr 26 '24 05:04 elasticmachine

Hey @yugalkaushik what are you looking for exactly? Is it related to this issue?

pierrehilbert avatar Apr 30 '24 07:04 pierrehilbert

Noo I was going through some issues but realised I need to go through the documentation of your organization to contribute with efficiency. So is there any medium I can learn and get an idea a.

On Tue, Apr 30, 2024, 12:58 PM Pierre HILBERT @.***> wrote:

Hey @yugalkaushik https://github.com/yugalkaushik what are you looking for exactly? Is it related to this issue?

— Reply to this email directly, view it on GitHub https://github.com/elastic/beats/issues/39146#issuecomment-2084585500, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBEHPRFIJTEPUDYNGAUTKS3Y75B25AVCNFSM6AAAAABGTNS7J6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBUGU4DKNJQGA . You are receiving this because you were mentioned.Message ID: @.***>

yugalkaushik avatar Apr 30 '24 08:04 yugalkaushik

Best would be to open another issue to discuss this and avoid bringing content not related to this issue here. You can start by looking at https://www.elastic.co/guide/en/beats/devguide/current/beats-contributing.html to get more information about how to contribute.

pierrehilbert avatar Apr 30 '24 09:04 pierrehilbert