fluent-plugin-kafka icon indicating copy to clipboard operation
fluent-plugin-kafka copied to clipboard

Support for wildcard `topic_key`

Open brsolomon-deloitte opened this issue 3 years ago • 4 comments

Is your feature request related to a problem? Please describe.

If I have a wildcard <match zeek.**> for @type kafka2, it would be great to have a way for topic_key to simply mirror the actual topic that is picked up.

Describe the solution you'd like

<match zeek.**>
  @type kafka2

...

  # write events to this kafka topic
  topic_key TODO # < HERE
  <buffer TODO>
    @type file
    path /var/log/td-agent/buffer/td
    flush_interval 10s
  </buffer>
  default_topic TODO

</match>

Describe alternatives you've considered

none

Additional context

none

brsolomon-deloitte avatar Nov 03 '21 14:11 brsolomon-deloitte

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

github-actions[bot] avatar Feb 02 '22 10:02 github-actions[bot]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

github-actions[bot] avatar May 04 '22 10:05 github-actions[bot]

This issue was automatically closed because of stale in 30 days

github-actions[bot] avatar Jun 03 '22 10:06 github-actions[bot]

Hey @brsolomon-deloitte, we should be able to achieve the same effect by using the topic parameter

e.g.

<match **>
   # etc
   topic "${tag}"

  <buffer tag>
    # etc
 </buffer>
</match>

The topic parameter is not currently documented but we'll work on that.

raytung avatar Aug 17 '22 06:08 raytung