kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-12829: Remove deprecated Topology#addGlobalStore of old Processor API

Open pegasas opened this issue 1 year ago • 12 comments

related to https://issues.apache.org/jira/browse/KAFKA-12829

remove: org.apache.kafka.streams.Topology#addGlobalStore

Committer Checklist (excluded from commit message)

  • [ ] Verify design and implementation
  • [ ] Verify test coverage and CI build status
  • [ ] Verify documentation (including upgrade notes)

pegasas avatar Aug 05 '24 07:08 pegasas

@pegasas -- Any updates on this PR?

mjsax avatar Aug 22 '24 19:08 mjsax

@pegasas -- Any updates on this PR?

org.apache.kafka.streams.Topology#addProcessor(java.lang.String, org.apache.kafka.streams.processor.ProcessorSupplier, java.lang.String...)

It looks like this method still uses by examples. shall we remove it in this loop? image

I shrink the scope of this commit so that we could merge little PR step by step

pegasas avatar Aug 25 '24 16:08 pegasas

There is two overloads. The old one, using org.apache.kafka.streams.processor.ProcessorSupplier<K, V>

org.apache.kafka.streams.Topology#addProcessor(java.lang.String, org.apache.kafka.streams.processor.ProcessorSupplier, java.lang.String...)

And the new one using org.apache.kafka.streams.processor.api.ProcessorSupplier<KIn, VIn, KOut, VOut>

org.apache.kafka.streams.Topology#addProcessor(java.lang.String, org.apache.kafka.streams.processor.ProcessorSupplier<KIn, VIn, KOut, VOut>, java.lang.String...)

Note the different package names. Only the old one is deprecated is should be removed, and as far as I can tell, it's only used in some test (which can also be removed).

The screenshot you posted seem to refer to the new one though.

However, it might be good to split into multiple PRs, and if this PR only removed addGlobalStore() it should be sufficient.

mjsax avatar Aug 25 '24 20:08 mjsax

The build has checkstyle errors:

> Task :streams:checkstyleMain
--
  | 2993 | 09:39:12 AM | [ant:checkstyle] [ERROR] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-16791/streams/src/main/java/org/apache/kafka/streams/processor/ProcessorContext.java:23:8: Unused import - org.apache.kafka.streams.Topology. [UnusedImports]
  | 2994 | 09:39:12 AM | [ant:checkstyle] [ERROR] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-16791/streams/src/main/java/org/apache/kafka/streams/processor/api/ProcessingContext.java:22:8: Unused import - org.apache.kafka.streams.Topology. [UnusedImports]

mjsax avatar Aug 25 '24 20:08 mjsax

The build has checkstyle errors:

> Task :streams:checkstyleMain
--
  | 2993 | 09:39:12 AM | [ant:checkstyle] [ERROR] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-16791/streams/src/main/java/org/apache/kafka/streams/processor/ProcessorContext.java:23:8: Unused import - org.apache.kafka.streams.Topology. [UnusedImports]
  | 2994 | 09:39:12 AM | [ant:checkstyle] [ERROR] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-16791/streams/src/main/java/org/apache/kafka/streams/processor/api/ProcessingContext.java:22:8: Unused import - org.apache.kafka.streams.Topology. [UnusedImports]

fixed these checkstyle errors. Is there any commands with gradle which I can run in local? like mvn spotless:apply or something?

pegasas avatar Aug 26 '24 02:08 pegasas

Yes, check the readme for gradle commands: https://github.com/apache/kafka

mjsax avatar Aug 26 '24 20:08 mjsax

There is still checkstyle errors.

mjsax avatar Aug 26 '24 21:08 mjsax

https://github.com/apache/kafka/blob/trunk/.github/workflows/build.yml#L74

image

image

It looks like origin command './gradlew checkstyleMain checkstyleTest spotlessCheck --info' not work now, and new py script also could not be executed in my linux notebook.

Not sure if it is in a stable status currently.

pegasas avatar Aug 27 '24 01:08 pegasas

From the build output:


FAILURE: Build completed with 2 failures.
--
  | 3850 | 07:23:42 PM |  
  | 3851 | 07:23:42 PM | 1: Task failed with an exception.
  | 3852 | 07:23:42 PM | -----------
  | 3853 | 07:23:42 PM | * What went wrong:
  | 3854 | 07:23:42 PM | Execution failed for task ':streams:spotlessJavaCheck'.
  | 3855 | 07:23:42 PM | > The following files had format violations:
  | 3856 | 07:23:42 PM | src/main/java/org/apache/kafka/streams/processor/api/ProcessingContext.java
  | 3857 | 07:23:42 PM | @@ -17,7 +17,6 @@
  | 3858 | 07:23:42 PM | package·org.apache.kafka.streams.processor.api;
  | 3859 | 07:23:42 PM |  
more_vert | 3860 | 07:23:42 PM | import·org.apache.kafka.common.serialization.Serde;
  | 3861 | 07:23:42 PM | -import·org.apache.kafka.streams.StreamsBuilder;
  | 3862 | 07:23:42 PM | import·org.apache.kafka.streams.StreamsMetrics;
  | 3863 | 07:23:42 PM | import·org.apache.kafka.streams.Topology;
  | 3864 | 07:23:42 PM | import·org.apache.kafka.streams.processor.Cancellable;
  | 3865 | 07:23:42 PM | Run './gradlew :streams:spotlessApply' to fix these violations.

If the readme is outdated, could you do a new PR to update it?

mjsax avatar Aug 28 '24 01:08 mjsax

From the build output:


FAILURE: Build completed with 2 failures.
--
  | 3850 | 07:23:42 PM |  
  | 3851 | 07:23:42 PM | 1: Task failed with an exception.
  | 3852 | 07:23:42 PM | -----------
  | 3853 | 07:23:42 PM | * What went wrong:
  | 3854 | 07:23:42 PM | Execution failed for task ':streams:spotlessJavaCheck'.
  | 3855 | 07:23:42 PM | > The following files had format violations:
  | 3856 | 07:23:42 PM | src/main/java/org/apache/kafka/streams/processor/api/ProcessingContext.java
  | 3857 | 07:23:42 PM | @@ -17,7 +17,6 @@
  | 3858 | 07:23:42 PM | package·org.apache.kafka.streams.processor.api;
  | 3859 | 07:23:42 PM |  
more_vert | 3860 | 07:23:42 PM | import·org.apache.kafka.common.serialization.Serde;
  | 3861 | 07:23:42 PM | -import·org.apache.kafka.streams.StreamsBuilder;
  | 3862 | 07:23:42 PM | import·org.apache.kafka.streams.StreamsMetrics;
  | 3863 | 07:23:42 PM | import·org.apache.kafka.streams.Topology;
  | 3864 | 07:23:42 PM | import·org.apache.kafka.streams.processor.Cancellable;
  | 3865 | 07:23:42 PM | Run './gradlew :streams:spotlessApply' to fix these violations.

If the readme is outdated, could you do a new PR to update it?

It seems not work on my local. I will dig if there is other solution. It seems there is no special action in .github/build.yml image

pegasas avatar Aug 28 '24 16:08 pegasas

It seems not work on my local

Why not fix it manually?

mjsax avatar Aug 28 '24 21:08 mjsax

It seems not work on my local

Why not fix it manually?

This is because ci pending costs too much time.

Done. It seems I can run ./gradlew --build-cache --info --scan check -x test to check style.

image

pegasas avatar Aug 29 '24 04:08 pegasas

This is because ci pending costs too much time.

Well, you should be able to reproduce checkstyle error locally -- "apply" means to auto-update the code, but just running ./gradlew checkstyleMain chechstyleTest spotbugsMain spotbugsTest should give you all errors locally and you could update the code manually to address them.

Anyway. Seems you found a way and the last build did not show any error :)

mjsax avatar Aug 29 '24 23:08 mjsax

Thanks for the PR. Merged to trunk.

mjsax avatar Aug 29 '24 23:08 mjsax