constellation
constellation copied to clipboard
bootstrapper: override Cilium MTU auto-detection on GCP (requires Cilium 1.13.0)
Proposed change(s)
- override Cilium MTU auto-detection on GCP
This should fix the issues with benchmarking in the CI, where Cilium picks up the MTU from Podman. On normal deployments this should not be an issue so far.
This requires Cilium 1.13.0 as the MTU value does not exist in Helm before, see: https://github.com/cilium/cilium/pull/20639
This workaround could be way easier by hardcoding the network interface to ens3
. However, not sure if Google will ever change that in the future, so let's just to it the tedious way I presume.
Draft for now since we need to upgrade to Cilium 1.13.0 first.
Related issue
- Cilium's auto detection picks up the wrong device: https://github.com/cilium/cilium/issues/14339
Checklist
- [ ] Update docs
- [ ] Add labels (e.g., for changelog category)
- [ ] Link to Milestone
Just a heads-up: This branch should be rebased before merge and bazel build files should be generated.
This document has up-to-date information on the required tasks (bazel run gazelle
).
Rebased. Gazelle should be happy without any changes since I didn't touch any imports.
It's been some time since this PR was opened and I don't think we have any performance issues due to wrong MTU settings on GCP anymore. There may have been a fix on Cilium's side (although the mentioned issue is still open), or something else changed, but the numbers below look decent to me, even more so compared with the other clouds.
Excerpt from https://github.com/edgelesssys/constellation/actions/runs/9524986833:
gcp-sev-snp
- Commit of current benchmark: 429711bf5db72908a725a52d5fc670527d0560ba
- Commit of previous benchmark: 32588339f345dd02b26dd9e85fa27b382257dad6
Benchmark suite | Metric | Current | Previous | Ratio |
---|---|---|---|---|
read_iops | iops (IOPS) | 5325.914 | 5326.2476 | 1.0 ⬆️ |
write_iops | iops (IOPS) | 5218.9717 | 5213.9775 | 1.001 ⬆️ |
read_bw | bw_kbytes (KiB/s) | 245757.0 | 246201.0 | 0.998 ⬇️ |
write_bw | bw_kbytes (KiB/s) | 237253.0 | 237102.0 | 1.001 ⬆️ |
pod2pod | tcp_bw_mbit (Mbit/s) | 4999.0 | 4698.0 | 1.064 ⬆️ |
pod2pod | udp_bw_mbit (Mbit/s) | 3704.0 | 3832.0 | 0.967 ⬇️ |
pod2svc | tcp_bw_mbit (Mbit/s) | 4995.0 | 4824.0 | 1.035 ⬆️ |
pod2svc | udp_bw_mbit (Mbit/s) | 3576.0 | 3832.0 | 0.933 ⬇️ |