Add New Environment Variables for Load Balancer Configuration
This PR introduces additional environment variables for load balancer configuration. These variables are designed to be set globally as defaults and can be overridden using annotations.
The main motivation is to improve support for GatewayAPI, as the Gateway annotation limit of 8 is restrictive and many settings are commonly needed across all load balancers from the same or even differen GatewayAPI providers. Additionally, this change allows environment-specific presets such as the new subnet IP range to be set globally. This removes the need to configure these settings in each service or use templating/patching to use the same service manifest for different environments.
New environment vars:
HCLOUD_LOAD_BALANCERS_ALGORITHM_TYPEHCLOUD_LOAD_BALANCERS_DISABLE_PUBLIC_NETWORKHCLOUD_LOAD_BALANCERS_HEALTH_CHECK_INTERVALHCLOUD_LOAD_BALANCERS_HEALTH_CHECK_RETRIESHCLOUD_LOAD_BALANCERS_HEALTH_CHECK_TIMEOUTHCLOUD_LOAD_BALANCERS_PRIVATE_SUBNET_IP_RANGEHCLOUD_LOAD_BALANCERS_TYPEHCLOUD_LOAD_BALANCERS_USES_PROXYPROTOCOL
Hey @M4t7e,
just to confirm, the "8 Annotations" limit is from the Gateway.spec.infrastructure.annotations field, right?
I found this thread where the limit was discussed and added, it sounds like this is a "soft" limit that can be raised if a reasonable case is made for more than 8 annotations: https://github.com/kubernetes-sigs/gateway-api/pull/1757#discussion_r1327843184
Hey @apricote,
yes, that's the limit I was referring to. I began researching ways to preset annotations in GatewayAPI, since the concept is not to have a single Gateway (Load Balancer) for everything, like it is often the case for Ingress Controller, but to have the flexibility of creating multiple Gateways. To avoid repeating the same config, I was looking into options setting global annotation setting. That's when I came across this issue: https://github.com/kubernetes-sigs/gateway-api/issues/2734
From what I understand, the annotation limit can only be increased through provider-specific implementations like Istio and Envoy already support. In my case, I’m planning to use Cilium GatewayAPI, which as far as I know doesn’t support adding annotations with their custom config.
Btw, this is the actual issue where we want to add GatewayAPI support: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/pull/216
I personally dislike the many annotations, and have often wondered how we could provide a better alternative.
The Gateway API parameterRef looks better, but a Gateway API Provider needs to implement L7 functionality, which our Load Balancer currently do not provide. So I figured that it makes no sense to built our own Gateway API Provider.
@lukasmetzner will be back next week to take a closer look at the MR.
Codecov Report
:x: Patch coverage is 77.96610% with 26 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 64.98%. Comparing base (4d970c6) to head (4572396).
:warning: Report is 1 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| internal/config/config.go | 65.85% | 7 Missing and 7 partials :warning: |
| internal/hcops/load_balancer.go | 84.41% | 10 Missing and 2 partials :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #1052 +/- ##
==========================================
- Coverage 68.43% 64.98% -3.46%
==========================================
Files 23 23
Lines 2531 2610 +79
==========================================
- Hits 1732 1696 -36
- Misses 629 746 +117
+ Partials 170 168 -2
| Flag | Coverage Δ | |
|---|---|---|
| e2e | ? |
|
| unit | 64.98% <77.96%> (+0.46%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
e2e test not passing is fine here. This is an expected permission issue.
@lukasmetzner I made the changes you suggested. Could you please take another look?
@lukasmetzner I made the changes you suggested. Could you please take another look?
@M4t7e The code looks and the e2e tests succeed locally on my machine. They are failing here due to expected permission issues with forks.
The only remaining thing is the Lint / pre-commit job, which should succeed here. Looking at the logs, only golangci-lint is failing with some minor refactors necessary: https://github.com/hetznercloud/hcloud-cloud-controller-manager/actions/runs/19902492646/job/57239368786?pr=1052
Could you please fix them, then we can merge this PR. You can either run golangci-lint run --fix directly or use pre-commit.
@lukasmetzner I updated the code formatting to comply with the latest golangci-lint version 2.7.1. This also required a change in internal/metrics/metrics.go:
❯ golangci-lint run
internal/metrics/metrics.go:24:1: File is not properly formatted (goimports)
"k8s.io/component-base/metrics/legacyregistry"
^
1 issues:
* goimports: 1
@lukasmetzner I updated the code formatting to comply with the latest
golangci-lintversion2.7.1. This also required a change ininternal/metrics/metrics.go:❯ golangci-lint run internal/metrics/metrics.go:24:1: File is not properly formatted (goimports) "k8s.io/component-base/metrics/legacyregistry" ^ 1 issues: * goimports: 1
golangci-lint seems to behave differently when --fix is added. You can ignore this here.