flink-remote-shuffle icon indicating copy to clipboard operation
flink-remote-shuffle copied to clipboard

Conflict with Flink 1.15.4

Open dangshazi opened this issue 1 year ago • 2 comments

Flink changes ConnectionID in 1.15.4. Related issue : #21358

Error msg:

java.lang.NoSuchMethodError: org.apache.flink.runtime.io.network.ConnectionID.<init>(Ljava/net/InetSocketAddress;I)V
	at com.alibaba.flink.shuffle.plugin.transfer.RemoteShuffleInputGate$FakedRemoteInputChannel.<init>(RemoteShuffleInputGate.java:726)
	at com.alibaba.flink.shuffle.plugin.transfer.RemoteShuffleInputGate.getChannel(RemoteShuffleInputGate.java:682)
	at org.apache.flink.runtime.io.network.partition.consumer.InputGate.setChannelStateWriter(InputGate.java:90)
	at org.apache.flink.runtime.taskmanager.InputGateWithMetrics.setChannelStateWriter(InputGateWithMetrics.java:120)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.injectChannelStateWriterIntoChannels(StreamTask.java:491)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:464)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:363)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:336)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:328)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:318)
	at org.apache.flink.streaming.runtime.tasks.AbstractTwoInputStreamTask.<init>(AbstractTwoInputStreamTask.java:50)
	at org.apache.flink.streaming.runtime.tasks.TwoInputStreamTask.<init>(TwoInputStreamTask.java:49)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1612)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:737)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:569)
	at java.lang.Thread.run(Thread.java:748)

dangshazi avatar Apr 26 '23 03:04 dangshazi

Yes that's right.

If you want to compile with new Flink 1.15, you can use the patch in the main branch. https://github.com/flink-extended/flink-remote-shuffle/commit/23f3fbfa2bfe264d86d1afe3aaceb1c3fd48e36a

TanYuxin-tyx avatar Apr 26 '23 03:04 TanYuxin-tyx

Thanks

dangshazi avatar Apr 26 '23 06:04 dangshazi