Kafka logger not working

[gaoxingliang]

Description

I try to use kafka logger to record all the requests and response and replace the clickhouse-logger(it's working) after I add it in the apisix dashboard, it shows nothing messages about kafka. and when I triggered a request and found the data is never sent to kafka.

at same time, I created another plugin response-rewrite and found it's working.

The apisix conf shows it's enabled:

plugins:    # plugin list
  - api-breaker
  - authz-keycloak
  - basic-auth
  - batch-requests
  - consumer-restriction
  - cors
  - echo
  - fault-injection
  - file-logger
  - grpc-transcode
  - grpc-web
  - hmac-auth
  - http-logger
  - ip-restriction
  - ua-restriction
  - jwt-auth
  - kafka-logger
  - key-auth
  - limit-conn
  - limit-count
  - limit-req
  - node-status
  - openid-connect
  - authz-casbin
  - prometheus
  - proxy-cache
  - proxy-mirror
  - proxy-rewrite
  - redirect
  - referer-restriction
  - request-id
  - request-validation
  - response-rewrite
  - serverless-post-function
  - serverless-pre-function
  - sls-logger
  - syslog
  - tcp-logger
  - udp-logger
  - uri-blocker
  - wolf-rbac
  - zipkin
  - traffic-split
  - gzip
  - real-ip
  - ext-plugin-pre-req
  - ext-plugin-post-req
  - ext-plugin-post-resp
  - server-info
  - elasticsearch-logger
  - clickhouse-logger
  - forward-auth
stream_plugins:
  - mqtt-proxy
  - ip-restriction
  - limit-conn

kafka logger conf:

{
  "_meta": {
    "disable": false
  },
  "batch_max_size": 1,
  "brokers": [
    {
      "host": "kafka.devops",
      "port": 9092
    }
  ],
  "disable": false,
  "kafka_topic": "apisix-log-test",
  "name": "kafka logger"
}

It seems this plugin is DISAPPEARED.

Environment

• APISIX version (run apisix version): 3.3.0
• APISIX dashboard: 3.0.0
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): 3.5.0
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[shreemaan-abhishek]

Are there any errors in error.log?

[gaoxingliang]

@shreemaan-abhishek I don;t see any error in the apisix error log. I created a new version cluster with 3.6.0. and it worked. and I also found the apisix occasionally got error of topic not found (even it's there in apisix 3.6.0):

2023/10/19 05:56:00 [error] 51#51: *1062684 [lua] batch-processor.lua:95: Batch Processor[apisix-kafkalogger] failed to process entries: failed to send data to Kafka topic: not found topic, brokers: null, context: ngx.timer, client: 10.233.113.0, server: 0.0.0.0:9080
2023/10/19 05:56:00 [error] 51#51: *1062684 [lua] batch-processor.lua:104: Batch Processor[apisix-kafkalogger] exceeded the max_retry_count[1] dropping the entries, context: ngx.timer, client: 10.233.113.0, server: 0.0.0.0:9080

[Revolyssup]

@gaoxingliang This might be an issue with lua-resty-kafka when FQDN is used instead of IP. Relevant issues: https://github.com/doujiang24/lua-resty-kafka/issues/6 https://github.com/doujiang24/lua-resty-kafka/issues/5

[gaoxingliang]

This issue persist. The apisix version is 3.6.0. and the conf:

{
  "_meta": {
    "disable": false
  },
  "brokers": [
    {
      "host": "192.168.102.227",
      "port": 39091
    }
  ],
  "include_req_body": true,
  "kafka_topic": "apisix-log-test",
  "name": "apisix-kafkalogger"
}

and the error log:

 2023/11/21 09:02:25 [warn] 55#55: *1345942 [lua] plugins.lua:54: query parameter "all" will be deprecated soon., client: 10.233.113.84, server: , request: "GET /apisix/admin/plugins?all=true HTTP/1.1", host: "apisix-admin.apisix.svc.cluster.local:9180"

 2023/11/21 09:02:28 [error] 57#57: *1350184 [lua] batch-processor.lua:95: Batch Processor[apisix-kafkalogger] failed to process entries: failed to send data to Kafka topic: not found broker, brokers: null, context: ngx.timer, client: 10.233.107.0, server: 0.0.0.0:9080

 2023/11/21 09:02:28 [error] 57#57: *1350184 [lua] batch-processor.lua:104: Batch Processor[apisix-kafkalogger] exceeded the max_retry_count[1] dropping the entries, context: ngx.timer, client: 10.233.107.0, server: 0.0.0.0:9080

I don't know why it said the not found broker. I telnet the kafka and it's connected.

I found the kafka logger plugin code error msg use the deprecated broker_list. so I also try with the conf:

  "broker_list": {
    "192.168.102.227": 39091
  },

It also says the broker list is null. I think the problem is: the conf get from etcd or somewhere is NULL. the source code is here: https://github.com/apache/apisix/blob/master/apisix/plugins/kafka-logger.lua#L203

[monkeyDluffy6017]

@gaoxingliang have you solved your problem?

[gaoxingliang]

@monkeyDluffy6017 no. It's unstable. I have to do the log manually.

[kayx23]

Please try the examples in this doc to see if anything helps with your case: https://docs.api7.ai/hub/kafka-logger

They were tested to be working and the doc is more current.

[github-actions[bot]]

This issue has been marked as stale due to 350 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.