incubator-streampark icon indicating copy to clipboard operation
incubator-streampark copied to clipboard

[Bug]Error while trying to split key and value in configuration

Open andystenhe opened this issue 10 months ago • 3 comments

What changes were proposed in this pull request

Issue Number: close #4199

Brief change log

fix Flink config.yaml Parsing Error​

Verifying this change

This change is already covered by existing tests, such as (please describe tests).

org.apache.streampark.common.util.PropertiesUtilsTestCase

Does this pull request potentially affect one of the following parts

  • Dependencies (does it add or upgrade a dependency): no

andystenhe avatar Mar 06 '25 11:03 andystenhe

hi:

Thanks for your contribution. Before Flink 1.19, the format of the flink-conf.yaml file did not adhere to the standard YAML specification. When parsing these parameters, Flink implemented its own custom parsing logic, which is also used in StreamPark. If a standard YAML parser were applied, it would fail to parse the file correctly.

This issue has been raised multiple times in the Flink community. Flink 2.0's file format was standardized to align with proper YAML specifications. However, this updated parsing method is only suitable for Flink 2.0 and later versions. For versions below 2.0, the original parsing approach must still be retained. Therefore, we cannot resolve this problem with a one-size-fits-all solution. Instead, we need to determine the Flink version and select the appropriate parsing method based on the version.

wolfboys avatar Mar 14 '25 07:03 wolfboys

fixed: https://github.com/apache/streampark/pull/4264

wolfboys avatar Jun 30 '25 08:06 wolfboys