hcloud-cloud-controller-manager icon indicating copy to clipboard operation
hcloud-cloud-controller-manager copied to clipboard

Add New Environment Variables for Load Balancer Configuration

Open M4t7e opened this issue 2 months ago • 8 comments

This PR introduces additional environment variables for load balancer configuration. These variables are designed to be set globally as defaults and can be overridden using annotations.

The main motivation is to improve support for GatewayAPI, as the Gateway annotation limit of 8 is restrictive and many settings are commonly needed across all load balancers from the same or even differen GatewayAPI providers. Additionally, this change allows environment-specific presets such as the new subnet IP range to be set globally. This removes the need to configure these settings in each service or use templating/patching to use the same service manifest for different environments.

New environment vars:

  • HCLOUD_LOAD_BALANCERS_ALGORITHM_TYPE
  • HCLOUD_LOAD_BALANCERS_DISABLE_PUBLIC_NETWORK
  • HCLOUD_LOAD_BALANCERS_HEALTH_CHECK_INTERVAL
  • HCLOUD_LOAD_BALANCERS_HEALTH_CHECK_RETRIES
  • HCLOUD_LOAD_BALANCERS_HEALTH_CHECK_TIMEOUT
  • HCLOUD_LOAD_BALANCERS_PRIVATE_SUBNET_IP_RANGE
  • HCLOUD_LOAD_BALANCERS_TYPE
  • HCLOUD_LOAD_BALANCERS_USES_PROXYPROTOCOL

M4t7e avatar Oct 20 '25 17:10 M4t7e

Hey @M4t7e,

just to confirm, the "8 Annotations" limit is from the Gateway.spec.infrastructure.annotations field, right?

I found this thread where the limit was discussed and added, it sounds like this is a "soft" limit that can be raised if a reasonable case is made for more than 8 annotations: https://github.com/kubernetes-sigs/gateway-api/pull/1757#discussion_r1327843184

apricote avatar Oct 24 '25 12:10 apricote

Hey @apricote,

yes, that's the limit I was referring to. I began researching ways to preset annotations in GatewayAPI, since the concept is not to have a single Gateway (Load Balancer) for everything, like it is often the case for Ingress Controller, but to have the flexibility of creating multiple Gateways. To avoid repeating the same config, I was looking into options setting global annotation setting. That's when I came across this issue: https://github.com/kubernetes-sigs/gateway-api/issues/2734

From what I understand, the annotation limit can only be increased through provider-specific implementations like Istio and Envoy already support. In my case, I’m planning to use Cilium GatewayAPI, which as far as I know doesn’t support adding annotations with their custom config.

Btw, this is the actual issue where we want to add GatewayAPI support: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/pull/216

M4t7e avatar Oct 24 '25 13:10 M4t7e

I personally dislike the many annotations, and have often wondered how we could provide a better alternative.

The Gateway API parameterRef looks better, but a Gateway API Provider needs to implement L7 functionality, which our Load Balancer currently do not provide. So I figured that it makes no sense to built our own Gateway API Provider.


@lukasmetzner will be back next week to take a closer look at the MR.

apricote avatar Oct 24 '25 14:10 apricote

Codecov Report

:x: Patch coverage is 77.96610% with 26 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 64.98%. Comparing base (4d970c6) to head (4572396). :warning: Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/config/config.go 65.85% 7 Missing and 7 partials :warning:
internal/hcops/load_balancer.go 84.41% 10 Missing and 2 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1052      +/-   ##
==========================================
- Coverage   68.43%   64.98%   -3.46%     
==========================================
  Files          23       23              
  Lines        2531     2610      +79     
==========================================
- Hits         1732     1696      -36     
- Misses        629      746     +117     
+ Partials      170      168       -2     
Flag Coverage Δ
e2e ?
unit 64.98% <77.96%> (+0.46%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Oct 28 '25 15:10 codecov[bot]

e2e test not passing is fine here. This is an expected permission issue.

lukasmetzner avatar Oct 28 '25 15:10 lukasmetzner

@lukasmetzner I made the changes you suggested. Could you please take another look?

M4t7e avatar Dec 03 '25 17:12 M4t7e

@lukasmetzner I made the changes you suggested. Could you please take another look?

@M4t7e The code looks and the e2e tests succeed locally on my machine. They are failing here due to expected permission issues with forks.

The only remaining thing is the Lint / pre-commit job, which should succeed here. Looking at the logs, only golangci-lint is failing with some minor refactors necessary: https://github.com/hetznercloud/hcloud-cloud-controller-manager/actions/runs/19902492646/job/57239368786?pr=1052

Could you please fix them, then we can merge this PR. You can either run golangci-lint run --fix directly or use pre-commit.

lukasmetzner avatar Dec 05 '25 11:12 lukasmetzner

@lukasmetzner I updated the code formatting to comply with the latest golangci-lint version 2.7.1. This also required a change in internal/metrics/metrics.go:

❯ golangci-lint run
internal/metrics/metrics.go:24:1: File is not properly formatted (goimports)
        "k8s.io/component-base/metrics/legacyregistry"
^
1 issues:
* goimports: 1

M4t7e avatar Dec 05 '25 12:12 M4t7e

@lukasmetzner I updated the code formatting to comply with the latest golangci-lint version 2.7.1. This also required a change in internal/metrics/metrics.go:

❯ golangci-lint run
internal/metrics/metrics.go:24:1: File is not properly formatted (goimports)
        "k8s.io/component-base/metrics/legacyregistry"
^
1 issues:
* goimports: 1

golangci-lint seems to behave differently when --fix is added. You can ignore this here.

lukasmetzner avatar Dec 15 '25 09:12 lukasmetzner