hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[SUPPORT] Apache Flink 2.0 in Hudi

Open stslingaprabu opened this issue 6 months ago • 0 comments

Description:

Apache Flink 2.0 was recently released with several major changes and improvements in the runtime, APIs, and ecosystem compatibility. As Flink is a critical component in many streaming data pipelines, upgrading to version 2.0 is important for users who want to take advantage of its performance enhancements, improved fault tolerance, unified batch/streaming APIs, and long-term support commitments. However, current versions of Apache Hudi do not yet support Flink 2.0, which blocks users from upgrading their Flink-based data pipelines. The current Flink integration in Hudi relies on APIs and internal behaviors that may have changed or been deprecated in Flink 2.0.

Request: We would like to request support for Apache Flink 2.0 in Hudi, including: Updating the Hudi Flink integration modules to be compatible with Flink 2.0 APIs. Ensuring all critical operations (e.g., write path, compaction, cleaning, and stream reads) continue to work as expected. Running integration tests on Flink 2.0 to validate stability and performance. Providing documentation or migration notes for users upgrading from Flink 1.x to 2.0 with Hudi.

Impact: Support for Flink 2.0 in Hudi will enable: Compatibility with the latest Flink LTS release. Use of Flink's new features such as Table Store, new source/sink APIs, and improved execution engine. Better alignment with the Apache Flink ecosystem and future-proofing Flink-based deployments. Please consider this as a feature enhancement for upcoming Hudi releases.

stslingaprabu avatar May 27 '25 09:05 stslingaprabu