DataflowJavaSDK icon indicating copy to clipboard operation
DataflowJavaSDK copied to clipboard

Error when reading from External Unbounded Source and Write to PubSubIO

Open fsalem opened this issue 9 years ago • 0 comments
trafficstars

Hi,

When I'm running a simple streaming pipeline which reads tweets and then adds them to PubSub topic, I got this exception.

Jan 26, 2016, 6:50:59 PM (6bc86c75060a0e4): Exception: java.lang.NullPointerException com.google.cloud.dataflow.sdk.runners.worker.StreamingModeExecutionContext.flushState(StreamingModeExecutionContext.java:236) com.google.cloud.dataflow.sdk.runners.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:534) com.google.cloud.dataflow.sdk.runners.worker.StreamingDataflowWorker.access$300(StreamingDataflowWorker.java:84) com.google.cloud.dataflow.sdk.runners.worker.StreamingDataflowWorker$5.run(StreamingDataflowWorker.java:428) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745)

Here is the main code: PCollection<String> input = p.apply(Read.named("ReadTweets").from(new TwitterUnboundedSource())); input.apply(PubsubIO.Write.named("WriteToPubSub").topic( "projects/rapid-stream-118713/topics/tweets"));

You can check the code: https://github.com/fsalem/dataflow-kafka-demo/blob/master/src/main/avro/dataflow/WriteToPubSubPipeline.java

fsalem avatar Jan 26 '16 17:01 fsalem