Add Referencing feature in TLS config to reduce the duplicated rows
Description
Adds reference feature to TLS config loading. Now we can use xxxx.tls.ref=<name> to simply ask TLSUtils to load TLS settings from the same config file.
- It is introduced to reduce the duplicated entries in some cases like PinotBroker.
- It is fully backwards compatible The old settings will still work; we just provide a new way to add config
Why do we need it
Right now if Broker is running inside a Kubernetes cluster, we may need 3 or 4 sets of TLS settings for it: broker https, broker gRPC secure, broker client.access.internal, and client.access.external . It makes the config file super long with duplicated entries.
With this PR we can simply write TLS settings once, and reference it when we need it.
Tests done
Added several unit tests to cover it.
Release Notes
- TLS settings in config can be set by using format like
xxx.tls.ref=<name>to reference a shared setting set bypinot.shared.tls.<name>
cc @apucher this will shorten broker config files a lot. Please have a look, thanks!
Codecov Report
Merging #9264 (13b363c) into master (718f41f) will decrease coverage by
7.26%. The diff coverage is5.88%.
:exclamation: Current head 13b363c differs from pull request most recent head c2cd41b. Consider uploading reports for the commit c2cd41b to get more accurate results
@@ Coverage Diff @@
## master #9264 +/- ##
============================================
- Coverage 68.75% 61.48% -7.27%
+ Complexity 4755 4538 -217
============================================
Files 1859 1847 -12
Lines 99129 98782 -347
Branches 15077 15040 -37
============================================
- Hits 68161 60741 -7420
- Misses 26076 33447 +7371
+ Partials 4892 4594 -298
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | 26.40% <5.88%> (-0.03%) |
:arrow_down: |
| unittests1 | 67.07% <ø> (-0.01%) |
:arrow_down: |
| unittests2 | ? |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...apache/pinot/broker/api/HttpRequesterIdentity.java | 28.57% <0.00%> (-57.15%) |
:arrow_down: |
| ...org/apache/pinot/broker/api/RequesterIdentity.java | 50.00% <0.00%> (-50.00%) |
:arrow_down: |
| ...va/org/apache/pinot/spi/utils/CommonConstants.java | 27.69% <ø> (ø) |
|
| ...roker/requesthandler/BaseBrokerRequestHandler.java | 56.59% <50.00%> (-10.87%) |
:arrow_down: |
| ...ot/broker/requesthandler/BrokerRequestHandler.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| ...pinot/controller/recommender/io/ConfigManager.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| .../org/apache/pinot/client/AggregationResultSet.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| ...ntroller/recommender/rules/impl/JsonIndexRule.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| ...troller/recommender/io/metadata/FieldMetadata.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| ...roller/recommender/rules/impl/BloomFilterRule.java | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| ... and 261 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
we sort-of have this ability already. the default TLS settings for each component (broker, controller, ...) should already be injected in every connector-specific config.
See for example, here: https://github.com/apache/pinot/blob/13b363c109e69d5069cc41490cc9ebc37078093a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/TlsIntegrationTest.java#L134. All the default properties, like controller.tls.keystore.type are automatically copied into the internal and external connector configs