beats icon indicating copy to clipboard operation
beats copied to clipboard

Read journal entries from all boots

Open belimawr opened this issue 1 year ago • 6 comments

Proposed commit message

Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments.

This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag.

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [ ] ~~I have made corresponding changes to the documentation~~
  • [ ] ~~I have made corresponding change to the default configuration files~~
  • [x] I have added tests that prove my fix is effective or that my feature works
  • [x] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

~~## Disruptive User Impact~~

Author's Checklist

  • [x] Ensure TestInputParsers from filebeat/input/journald/input_parsers_test.go is not flaky
  • [x] Add tests to ensure messages from all boots are read
  • [ ] Check whether the restart of journalclt can cause problems with multiline parser (see https://github.com/elastic/beats/issues/41331)

How to test this PR locally

1. Run the tests

cd filebeat/input/journald
go test -run=TestInputCanReadAllBoots

2. Run Filebeat reading filebeat/input/journald/testdata/multiple-boots.journal

There must be 6 entries, you can see the plaintext entries by looking at filebeat/input/journald/testdata/multiple-boots.export or by running:

journalctl --file filebeat/input/journald/testdata/multiple-boots.export

3. Fully manual test

  1. Run the journald input on a machine (or a journal file) that has got messages from more than one boot
  2. Ensure all messages in the journal are correctly ingested.

Related issues

  • Closes https://github.com/elastic/beats/issues/41083

~~## Use cases~~ ~~## Screenshots~~ ~~## Logs~~

belimawr avatar Oct 15 '24 19:10 belimawr

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 41083-journald-input-all-boots upstream/41083-journald-input-all-boots
git merge upstream/main
git push upstream 41083-journald-input-all-boots

mergify[bot] avatar Oct 15 '24 19:10 mergify[bot]

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @belimawr? 🙏. For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit

mergify[bot] avatar Oct 15 '24 19:10 mergify[bot]

backport-8.x has been added to help with the transition to the new branch 8.x. If you don't need it please use backport-skip label and remove the backport-8.x label.

mergify[bot] avatar Oct 15 '24 19:10 mergify[bot]

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 41083-journald-input-all-boots upstream/41083-journald-input-all-boots
git merge upstream/main
git push upstream 41083-journald-input-all-boots

mergify[bot] avatar Oct 18 '24 21:10 mergify[bot]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

elasticmachine avatar Oct 18 '24 21:10 elasticmachine

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 41083-journald-input-all-boots upstream/41083-journald-input-all-boots
git merge upstream/main
git push upstream 41083-journald-input-all-boots

mergify[bot] avatar Oct 22 '24 23:10 mergify[bot]