zookeeper and output_data_type are not used

[PinkFloydVN]

trafficstars

Describe the bug

Hi team, After upgrade td-agent from 1.2.6 to 4.3.0, zookeeper and output_data_type in my td-agent template are not used. I searched on the internet but there is not much clear documents regarding alternatives for them when running td-agent 4.3.0. Would someone please help to take a look? Thank you.

To Reproduce

Include zookeeper and output_data_type in the td-agent template

Expected behavior

I expected that zookeeper and output_data_type should work.

Your Environment

- TD Agent version:
td-agent 4.3.0 fluentd 1.14.3 (438a82aead488a86180cd484bbc4e7e344a9032b)

- Operating system:
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

- Kernel version:
5.4.0-99-generic

Your Configuration

{% if match_tags_added %}
<match {{ tag }}>
{% end %}

  @type kafka2
  @id {{ ('kafka2_'+tag).replace('.**','').replace('.*','') }}
  <format>
    @type json
  </format>
  zookeeper {{ sysconfig.networking.zookeeper_connect_string }}

  {# possibly unused in latest version of plugin. #}
  client_id  {{ client_id }}

  default_topic {{ topic }}
  default_partition_key 0
  output_data_type json
  required_acks 1

  {{ include_file('tdagent/common-buffer.template',
                  tag=tag.replace('.**','').replace('.*',''),
                  buffer_sub=sysconfig.fluentd.kafka_buffer_directory,
                  buffer_queue_len=sysconfig.fluentd.kafka_buffer_queue_limit,
                  ) }}
{% if match_tags_added %}
</match>
{% end %}

Your Error Log

2022-02-16 09:54:28 +0000 [warn]: parameter 'zookeeper' in <match handle_request.url_request.**>
  @type kafka2
  @id kafka2_handle_request.url_request
  zookeeper localhost:1234
  client_id "user"
  default_topic "user_url_request"
  default_partition_key "0"
  output_data_type json
  required_acks 1
  <format>
    @type "json"
  </format>
  <buffer>
    @type "file"
    path "/my/kafka/path/handle_request.url_request/"
    chunk_limit_size 32m
    queue_limit_length 128
    queued_chunks_limit_size 4
    disable_chunk_backup true
    flush_interval 0.1s
    flush_at_shutdown false
    retry_forever true
    retry_wait 1s
    retry_max_interval 120s
    overflow_action drop_oldest_chunk
    flush_thread_count 2
  </buffer>
</match> is not used.
2022-02-16 09:54:28 +0000 [warn]: parameter 'output_data_type' in <match handle_request.url_request.**>
  @type kafka2
  @id kafka2_handle_request.url_request
  zookeeper localhost:1234
  client_id "user"
  default_topic "user_url_request"
  default_partition_key "0"
  output_data_type json
  required_acks 1
  <format>
    @type "json"
  </format>
  <buffer>
    @type "file"
    path "/my/kafka/path/handle_request.url_request/"
    chunk_limit_size 32m
    queue_limit_length 128
    queued_chunks_limit_size 4
    disable_chunk_backup true
    flush_interval 0.1s
    flush_at_shutdown false
    retry_forever true
    retry_wait 1s
    retry_max_interval 120s
    overflow_action drop_oldest_chunk
    flush_thread_count 2
  </buffer>
</match> is not used.

Additional context

No response

[PinkFloydVN]

Hi Ashie, Is there any update for this? Thank you

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

[github-actions[bot]]

This issue was automatically closed because of stale in 30 days