DRAFT: Enable status reporter for log inputs

[VihasMakwana]

[mergify[bot]]

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @VihasMakwana? 🙏. For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[mergify[bot]]

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add-log-status-reporter upstream/add-log-status-reporter
git merge upstream/main
git push upstream add-log-status-reporter

[VihasMakwana]

@belimawr can you take a look if you find some time?

[pierrehilbert]

@belimawr is now in PTO. @rdner could you please have a look here?

[pierrehilbert]

The log input is deprecated since 7.16.0.

@nimarezainia what is your opinion, should we do it for Log Input or consider closing this PR as this is deprecated?

I don't think we should spend any time on extending its functionality and testing. Instead I'd expect status reporting to be introduced in the filestream input.

FYI we already have it merged for Filestream input: https://github.com/elastic/beats/pull/40121

[jlind23]

I do agree with @rdner statement above, if it is deprecated this would be another reason why users need to switch to filestream. Also another reason why we need to migrate the container input.

[VihasMakwana]

I do agree with @rdner statement above, if it is deprecated this would be another reason why users need to switch to filestream. Also another reason why we need to migrate the container input.

@jlind23 Agree. But container input uses log input under the hood. If we add the status reporting to log input, it would be added for container input as well

[jlind23]

@jlind23 Agree. But container input uses log input under the hood. If we add the status reporting to log input, it would be added for container input as well

That is why my suggestion was rather to close this issue and spend time on migrating the container input to rely on filestream.

[VihasMakwana]

That is why my suggestion was rather to close this issue and spend time on migrating the container input to rely on filestream.

@jlind23 Oh, sorry. I misunderstood. I thought you were referring to add StatusReporter to container.

[jlind23]

@VihasMakwana @rdner as spoke over slack, this has to be delivered even if the log input is marked as deprecated. So @VihasMakwana please reply back to @rdner's comment and let's continue the review process here.

[nimarezainia]

if the PR is ready and tested, why not go through with it? doesn't seem like a risky endeavor.

We should have made this decision before pursuing the PR IMO. but now it's here what doe we lose in merging? many are still using 7.16 and on logs input.

[pierrehilbert]

@nimarezainia this is already merged. However, we are waiting for 8.15.1 before merging it into 8.15 branch as the time frame is now too short to merge it for 8.15.0.

[cmacknz]

doesn't seem like a risky endeavor.

The risk is we suddenly have the majority of agents report themselves as degraded and get a flood of support cases for things we could have identified and fixed ourselves if we let this change soak longer.