constellation icon indicating copy to clipboard operation
constellation copied to clipboard

bootstrapper: override Cilium MTU auto-detection on GCP (requires Cilium 1.13.0)

Open Nirusu opened this issue 1 year ago • 2 comments

Proposed change(s)

  • override Cilium MTU auto-detection on GCP

This should fix the issues with benchmarking in the CI, where Cilium picks up the MTU from Podman. On normal deployments this should not be an issue so far.

This requires Cilium 1.13.0 as the MTU value does not exist in Helm before, see: https://github.com/cilium/cilium/pull/20639

This workaround could be way easier by hardcoding the network interface to ens3. However, not sure if Google will ever change that in the future, so let's just to it the tedious way I presume.

Draft for now since we need to upgrade to Cilium 1.13.0 first.

Related issue

  • Cilium's auto detection picks up the wrong device: https://github.com/cilium/cilium/issues/14339

Checklist

  • [ ] Update docs
  • [ ] Add labels (e.g., for changelog category)
  • [ ] Link to Milestone

Nirusu avatar Mar 07 '23 17:03 Nirusu

Just a heads-up: This branch should be rebased before merge and bazel build files should be generated. This document has up-to-date information on the required tasks (bazel run gazelle).

malt3 avatar Mar 09 '23 15:03 malt3

Rebased. Gazelle should be happy without any changes since I didn't touch any imports.

Nirusu avatar Mar 09 '23 16:03 Nirusu

It's been some time since this PR was opened and I don't think we have any performance issues due to wrong MTU settings on GCP anymore. There may have been a fix on Cilium's side (although the mentioned issue is still open), or something else changed, but the numbers below look decent to me, even more so compared with the other clouds.

Excerpt from https://github.com/edgelesssys/constellation/actions/runs/9524986833:

gcp-sev-snp

Benchmark suite Metric Current Previous Ratio
read_iops iops (IOPS) 5325.914 5326.2476 1.0 ⬆️
write_iops iops (IOPS) 5218.9717 5213.9775 1.001 ⬆️
read_bw bw_kbytes (KiB/s) 245757.0 246201.0 0.998 ⬇️
write_bw bw_kbytes (KiB/s) 237253.0 237102.0 1.001 ⬆️
pod2pod tcp_bw_mbit (Mbit/s) 4999.0 4698.0 1.064 ⬆️
pod2pod udp_bw_mbit (Mbit/s) 3704.0 3832.0 0.967 ⬇️
pod2svc tcp_bw_mbit (Mbit/s) 4995.0 4824.0 1.035 ⬆️
pod2svc udp_bw_mbit (Mbit/s) 3576.0 3832.0 0.933 ⬇️

burgerdev avatar Jun 20 '24 08:06 burgerdev