[Logs - Journald] Allow multiple tailers on the same Journal

[gh123man]

What does this PR do?

This change allows multiple tailers to read the same journal. This is particularly useful when users want to apply different tags to different units from the same journal. This change also introduces the config_id setting to the journald tailer config. The config_id is used to uniquely identify the configuration so the agent can pick up where it left off in the event of a restart.

Motivation

Feature request. - Allow users to have more flexibility when configuring journald tailers.

Additional Notes

Possible Drawbacks / Trade-offs

Adding a new config key to the agent config surface. Additional complexity - but this is an optional key for a very specific use case. The alternatives are to hash (some of or all of) the config and use that as an identifier, but that would make it very difficult to make future changes.

The config_id setting is entirely optional and only used if you have multiple configs pointing to the same journal. NOTE: this setting is not require if the config specifies a path to a different journal.

Describe how to test/QA your changes

1. Create a logs config with multiple journald config entries:

logs:
  - type: journald
    container_mode: true
    source: foo
    service: foo
    include_units:
    - random-logger1.service
  - type: journald
    config_id: config1
    container_mode: true
    source: bar
    service: bar
    include_units:
    - random-logger2.service
  - type: journald
    config_id: config2
    container_mode: true
    source: abc
    service: abc
    include_units:
    - random-logger3.service

2. Create multiple systemd units that produce logs (with the names above random-loggerX.service
3. check agent status - confirm that all sources are reading bytes from the tailer
4. run agent stream-logs check that each unit is filtered correctly and no logs are duplicated between sources. Also confirm that the source/service is set correctly for each log line.

Reviewer's Checklist

• [ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
• [ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
• [ ] A release note has been added or the changelog/no-changelog label has been applied.
• [ ] Changed code has automated tests for its functionality.
• [ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
• [ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
• [ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
• [ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
• [ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
• [ ] If applicable, the config template has been updated.