inlong icon indicating copy to clipboard operation
inlong copied to clipboard

[Bug][DataProxy] DataProxy is not sending data to Pulsar

Open Gotodie opened this issue 2 years ago • 9 comments

What happened

dp is not sending data to pulsar. image image

What you expected to happen

There is a record of data consumption in pulsar, and the last data generated in clickhouse.

How to reproduce

When I finished the approval and kept adding data to the local source file, no data was generated in clickhouse. Finally, after finding the problem, I found that there was no data in pulsar. image image image

Environment

centos 7

InLong version

master

InLong Component

InLong DataProxy

Are you willing to submit PR?

  • [X] Yes, I am willing to submit a PR!

Code of Conduct

Gotodie avatar Jul 19 '22 03:07 Gotodie

Message has been sent to dp? and If the message has been sent to dp, please confirm whether the expected topic configuration information is contained in conf/topics.properties,

baomingyu avatar Jul 19 '22 06:07 baomingyu

In addition, please check whether the log contains the following information。 image image

baomingyu avatar Jul 19 '22 06:07 baomingyu

@baomingyu image image There is this error in inlong-dataproxy/logs/. image

Gotodie avatar Jul 19 '22 06:07 Gotodie

Please check if this topic exists in the pulsar cluster. and check the initial configuration steps of the cluster. If the initial configuration information of the cluster in the Manager is incorrect, the topic may not be created in advance.

baomingyu avatar Jul 20 '22 08:07 baomingyu

@baomingyu There is a corresponding topic in pulsar, but the topic always shows no data consumption

Gotodie avatar Jul 20 '22 08:07 Gotodie

this issue may be the same with https://github.com/apache/inlong/issues/5163, and we will look into it.

dockerzhang avatar Jul 21 '22 11:07 dockerzhang

fixed in https://github.com/apache/inlong/pull/5189. Please check again.

dockerzhang avatar Jul 24 '22 08:07 dockerzhang

When I use the latest code, I still can't get the data to pulsar, and new errors occurs. agent_errorlog managererrorlog

Gotodie avatar Jul 25 '22 07:07 Gotodie

这个问题你是否已经解决?我碰到了同样的问题,束手无策。1.4.0版本

ly100120 avatar Sep 15 '22 12:09 ly100120

This issue is stale because it has been open for 60 days with no activity.

github-actions[bot] avatar Nov 15 '22 02:11 github-actions[bot]

@ly100120 please add the steps to reproduce the issue.

dockerzhang avatar Nov 22 '22 08:11 dockerzhang