inlong icon indicating copy to clipboard operation
inlong copied to clipboard

[INLONG-11349][Sort] Integrate opentelemetry for sort-connectors-v1.15

Open qy-liuhuo opened this issue 1 year ago • 1 comments

Fixes #11349

Motivation

Integrate opentelemetry for sort-connectors-v1.15

Modifications

Integrate opentelemetry for:

  • sort-connector-iceberg-v1.15
  • sort-connector-kafka-v1.15
  • sort-connector-mysql-cdc-v1.15
  • sort-connector-pulsar-v1.15

However, some connectors that do not conform to the FLIP-27 have not yet been integrated

Verifying this change

(Please pick either of the following options)

  • [ ] This change is a trivial rework/code cleanup without any test coverage.

  • [ ] This change is already covered by existing tests, such as: (please describe tests)

  • [ ] This change added tests and can be verified as follows:

    (example:)

    • Added integration tests for end-to-end deployment with large payloads (10MB)
    • Extended integration test for recovery after broker failure

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

qy-liuhuo avatar Oct 13 '24 09:10 qy-liuhuo

When integrating sort-connector-kafka-v1.15 and sort-connector-pulsar-v1.15, the unit test fails. I currently think it is a Maven dependency issue, which I will try to resolve later.

qy-liuhuo avatar Oct 21 '24 06:10 qy-liuhuo

Thanks for your contribution. Sorry, but If we don't use the open telemetry, why do we initialize it in the open method? Maybe adding a new configuration to enable open telemetry is better?

aloyszhang avatar Nov 24 '24 04:11 aloyszhang

Thanks for your contribution. Sorry, but If we don't use the open telemetry, why do we initialize it in the open method? Maybe adding a new configuration to enable open telemetry is better?

Sorry, I haven't brought my computer recently. I will solve this problem latter

qy-liuhuo avatar Nov 28 '24 05:11 qy-liuhuo

Thanks for your contribution. Sorry, but If we don't use the open telemetry, why do we initialize it in the open method? Maybe adding a new configuration to enable open telemetry is better?

I have finished it, now we can control whether to enable log reporting through command line parameters when submitting flink tasks. Just add --enable.log.report true when submitting the task.

qy-liuhuo avatar Dec 10 '24 13:12 qy-liuhuo