beats icon indicating copy to clipboard operation
beats copied to clipboard

[Filebeat/Journald] allow to use a chroot when calling `journalctl`

Open belimawr opened this issue 2 weeks ago • 4 comments

Proposed commit message

Add support in the journald inpur for using chroot when calling
`journalctl`. In a container environment this allows to mount the host
file system into the container and use its `journalctl`, which
prevents any sort of incompatibility between the `journalctl` in the
container image and the host Journald.

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I have made corresponding changes to the documentation
  • [x] I have made corresponding change to the default configuration files
  • [x] I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
  • [x] I have added an entry in ./changelog/fragments using the changelog tool.

~~## Disruptive User Impact~~

Author's Checklist

  • [x] Finish manual testing steps

How to test this PR locally

Run the tests

cd filebeat
go test -v -count=1 -tags=integration -run=TestNewFactoryChroot ./input/journald/pkg/journalctl
go test -tags=integration -v -count=1 -run=TestJournaldChroot ./tests/integration

Manual testing

1. Create a filebeat.yml

filebeat.yml

filebeat.inputs:
  - type: journald
    id: journald-input-id
    chroot: /host
    journalctl_path: /usr/bin/journalctl

output.discard:
  enabled: true

# output.console:
#   enabled: true
#   pretty: false

logging:
  to_stderr: true
  metrics.period: 5s

2. Build a staicaly linked Filebeat binary

CGO_ENABLED=0 go build -ldflags '-extldflags "-static" -s' -tags timetzdata .

3. Create a Dockerfile and build a test image

Dockerfile

FROM alpine:latest AS prep

FROM scratch

COPY --from=prep /etc/alpine-release /etc/alpine-release
COPY filebeat /
COPY filebeat.yml /
ENTRYPOINT ["/filebeat"]
CMD ["-c", "/filebeat.yml", "--strict.perms=false"]
docker build -t filebeat-chroot-test .

4. Run the container

docker run --rm -v /:/host filebeat-chroot-test

You should see in the logs the journalctl starting:

{
  "@timestamp": "2025-12-10T17:10:04.317Z",
  "ecs.version": "1.6.0",
  "id": "journald-input-id",
  "input_id": "journald-input-id",
  "input_source": "LOCAL_SYSTEM_JOURNAL",
  "log.level": "info",
  "log.logger": "input.journald.reader.journalctl-runner",
  "log.origin": {
    "file.line": 152,
    "file.name": "journalctl/journalctl.go",
    "function": "github.com/elastic/beats/v7/filebeat/input/journald/pkg/journalctl.NewFactory.func1"
  },
  "message": "Journalctl command: journalctl --utc --output=json --no-pager --all --follow --no-tail --boot all",
  "path": "LOCAL_SYSTEM_JOURNAL",
  "service.name": "filebeat"
}
{
  "@timestamp": "2025-12-10T17:10:04.317Z",
  "ecs.version": "1.6.0",
  "id": "journald-input-id",
  "input_id": "journald-input-id",
  "input_source": "LOCAL_SYSTEM_JOURNAL",
  "log.level": "info",
  "log.logger": "input.journald.reader.journalctl-runner",
  "log.origin": {
    "file.line": 158,
    "file.name": "journalctl/journalctl.go",
    "function": "github.com/elastic/beats/v7/filebeat/input/journald/pkg/journalctl.NewFactory.func1"
  },
  "message": "journalctl started with PID 23",
  "path": "LOCAL_SYSTEM_JOURNAL",
  "service.name": "filebeat"
}

You can also see the number events published in the metrics that are logged every 5s.

If you want to see the events, remove the discard output and uncomment the console output in filebeat.yml, then rebuild the docker image and run it again.

Related issues

  • Closes https://github.com/elastic/beats/issues/47323

~~## Use cases~~ ~~## Screenshots~~ ~~## Logs~~

belimawr avatar Dec 09 '25 17:12 belimawr

:robot: GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

github-actions[bot] avatar Dec 09 '25 17:12 github-actions[bot]

The failing test on Windows is unrelated to this PR. The flaky test issue: https://github.com/elastic/beats/issues/48009

belimawr avatar Dec 09 '25 20:12 belimawr

🔍 Preview links for changed docs

github-actions[bot] avatar Dec 10 '25 22:12 github-actions[bot]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

elasticmachine avatar Dec 11 '25 13:12 elasticmachine

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b journald-chroot upstream/journald-chroot
git merge upstream/main
git push upstream journald-chroot

mergify[bot] avatar Dec 16 '25 23:12 mergify[bot]

This looks like it will work if you are root, have you found what the least privilege way to execute journalctl is yet? Can you for example be a non-root user with CAP_SYS_CHROOT (you might need more than just this) and execute journalctl?

The wolfi images do not have root as the running container user for example, so ideally we can keep that and add the minimum capabilities rather than being root if possible.

cmacknz avatar Dec 17 '25 21:12 cmacknz

This looks like it will work if you are root, have you found what the least privilege way to execute journalctl is yet? Can you for example be a non-root user with CAP_SYS_CHROOT (you might need more than just this) and execute journalctl?

The wolfi images do not have root as the running container user for example, so ideally we can keep that and add the minimum capabilities rather than being root if possible.

I've been trying different variations of privileges and doing some research, the only way I can get the chroot to work is when setting the user to root. Even without adding --cap-add CAP_SYS_CHROOT it works if I ran Filebeat as root (--user 0). Everything I found also states that using chroot requires root.

--cap-drop CAP_SYS_CHROOT prevents even root from running the chroot. So CAP_SYS_CHROOT is definitely required, but it seems to be enabled by default on my machine :man_shrugging:.

I'll update the docs to reflect it.

belimawr avatar Dec 18 '25 17:12 belimawr

The CI failure is unrelated to this PR, here is the flaky test isuse: https://github.com/elastic/beats/issues/48187

belimawr avatar Dec 18 '25 19:12 belimawr