flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-34498][filesystem] GSFileSystemFactory should not log full Flink log

Open jeyhunkarimov opened this issue 1 year ago • 2 comments

What is the purpose of the change

Avoid GSFileSystemFactory to log sensitive configurations

Brief change log

  • Hide sensitive values and add test

Verifying this change

Test: org.apache.flink.fs.gs.GSFileSystemFactoryTest::testSkipSensitiveConf()

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no )
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)

jeyhunkarimov avatar Feb 22 '24 20:02 jeyhunkarimov

CI report:

  • ad8c47d11bddac716039797ff76502560a2dcb3b Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Feb 22 '24 21:02 flinkbot

Hi @zentol could you please review this PR in your available time? Thanks

jeyhunkarimov avatar Feb 24 '24 21:02 jeyhunkarimov

See my comment in FLINK-34498. PR #24370 seems to fix the issue in a broader scale.

XComp avatar Feb 28 '24 14:02 XComp

Yes, you are right. I haven't checked https://github.com/apache/flink/pull/24370

jeyhunkarimov avatar Feb 28 '24 14:02 jeyhunkarimov

Somewhat. There's no good reason to the log the entire config in the first place because it's already logged on start-up, so it also creates a bunch of noise.

@zentol had a good point in his PR that we could actually remove the log statement all together because the config is already logged at during startup.

Could you remove the log statement in this PR and create a 1.19 backport?

XComp avatar Feb 28 '24 15:02 XComp

Definitely makes sense. I updated the PR and submitted backport to 1.19 (https://github.com/apache/flink/pull/24408)

jeyhunkarimov avatar Feb 28 '24 15:02 jeyhunkarimov