flink
flink copied to clipboard
[FLINK-33988][configuration] Fix the invalid configuration when using initialized root logger level on yarn deployment mode
What is the purpose of the change
Fix the invalid configuration when using initialized root logger level on yarn deployment mode
Brief change log
- Fill the containerized env values of root logger level before submiting yarn application.
Verifying this change
This change added tests and can be verified as follows:
- *Manually verified the change by running a 4 nodes cluster.
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with
@Public(Evolving)
: (yes / no) - The serializers: (yes / no / don't know)
- The runtime per-record code paths (performance sensitive): (yes / no / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
- The S3 file system connector: (yes / no / don't know)
Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
CI report:
- a746744c7e1e14d077316264c583c71bc6c6278d Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:-
@flinkbot run azure
re-run the last Azure build
Hi @1996fanrui, I've verified that the current release of Flink supports modifying the logger level in both Native K8s and Standalone mode. Since YARN prepares the container launching context in a special way, we'll need to do some special adaptation anyway. LGTM
Thanks @X-czh for the double-check and @1996fanrui review.
I made a fix like the env-value passing-in at https://github.com/RocMarshal/flink/commit/624c199e756ed5f04f849e8e53033bc1ef258f93
Would we merge the additional fix into the current pr ? Please let me know what's your opinion~ :)
I made a fix like the env-value passing-in at https://github.com/RocMarshal/flink/commit/624c199e756ed5f04f849e8e53033bc1ef258f93 Would we merge the additional fix into the current pr ? Please let me know what's your opinion~ :)
Same PR is fine for me, but separate JIRA and commits are necessary.
Separate JIRA and commits are easy to follow up and cherry pick for other users.