Drop Flink 1.18 support to reduce code duplication and improve testability

[arnaud-daroussin]

Hi @novakov-alexey,

You told me a few weeks ago you were looking for a way to improve testing for Flink1 and Flink2 without duplicating the tests.

I think the way to improve testability is to reduce code duplication. If we duplicate the code, we also need to duplicate associated tests.

I looked at the code to understand why so many classes have been duplicated between Flink 1.x and 2.x. I'm seeing many methods and APIs that were deprecated in Flink 1.19 and removed in Flink 2.0, so you had to duplicate the code to handle old APIs of Flink 1.18 where the replacement is not yet available and duplicate the code to handle the new APIs of Flink 2.0 when the old ones have been removed.

So if we drop Flink 1.18 support, we can reunify classes that were duplicated due to:

• the deprecation of ExecutionConfig in favor of SerializerConfig,
• the deprecation of Time in favor of Duration,
• the deprecation of Configuration in favor of OpenContext,
• the deprecation of resolveSchemaCompatibility(newSerializer: TypeSerializer[T]) in favor of resolveSchemaCompatibility(oldSerializerSnapshot: TypeSerializerSnapshot[T]) in TypeSerializerSnapshot.

It means we could reunify almost all the code, most notably all the typeinfo, serializers, snapshots, LowPrioImplicits. There is only some API that cannot be reunified like CoGroupedStreams or ScalaAllWindowFunctionWrapper because of a breaking change in Flink 2.0 without proper replacement before.

What do you think of this?

[novakov-alexey]

Hi @arnaud-daroussin

I am of course in favour of this idea. Let's wait until Flink 1.21 is out so that we will support last 3 Flink 1 versions: 1.21, .1.20, 1.19. Then we can drop Flink 1.18

[arnaud-daroussin]

You are very conservative 😄 !

In my opinion we should keep 1.19, 1.20 and 2.0. I think Flink will never release a 1.21, new features will be only on 2.x now.

[novakov-alexey]

Yes, just want to keep this idea of supported version. Perhaps someone relies on it :-) It may be true that 1.21 will not be released. Let's wait a bit and if it is so, we can drop support of Flink 1.18 and deduplicate the code. I am looking forward to this.

[arnaud-daroussin]

I checked the active branches on Flink, there is new commits only for fix and library update on release-1.19, release-1.20 and release-2.0.

master has version 2.1-SNAPSHOT.

I see no branch to prepare a version 1.21 and if they want to add a new feature on top of Flink 1.20 they will have to create a branch from release-1.20 to prepare it.

I think they follow the classic versioning philosophy to develop new feature only on master.

But of course we can wait, there is no hurry.

[novakov-alexey]

Thanks for checking. I hope this gonna stay like this, so that there will be no 1.21 released. We will then consider the Flink 2.0 as the third version we start to support.

[novakov-alexey]

@arnaud-daroussin

In my opinion we should keep 1.19, 1.20 and 2.0.

I think we are free to go with this idea. Do you want to try? I am looking forward to get rid of code duplication.

[arnaud-daroussin]

Hi @novakov-alexey, yes I'm taking the point.

[arnaud-daroussin]

Fixed with https://github.com/flink-extended/flink-scala-api/pull/275

[vbabenkoru]

@arnaud-daroussin I think the readme should be updated, it still states: Three latest of Apache Flink 1.x and one Flink 2.y versions are supported. While I think 1.19 and 1.20 are two latest versions.