mina

mina copied to clipboard

Preliminary Checks

• [X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/MinaProtocol/mina/issues
• [X] This issue is not a question, feature request, RFC, or anything other than a bug report. Please post those things in GitHub Discussions: https://github.com/MinaProtocol/mina/discussions

Summary

Sometimes the integrations tests pods end up on the same k8s node and try to allocate a so called hostPort. See an example below, but it's definitely not the only one doing this.

https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/archive-node/values.yaml#L24-L26

This of course ends up failing since only one pods will be capable of allocating the hostPort. I've seen this scenario happen on the google console.

Steps to replicate this Issue

1. Start a run on buildkite
2. Observe one of the integration test pods on Google Cloud console
3. See that some pods won't get scheduled as they can't allocate the port

Console output

PodUnschedulable

Reason
    Cannot schedule pods: node(s) didn't have free ports for the requested pod ports. 
    Learn more

Platform

Platform: Google Kubernetes Engine

Build link(s)

No response

Other manifests that seem to use hostPort.

https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/seed-node/values.yaml#L27 https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/seed-node/templates/seed-node.yaml#L140 https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/plain-node/templates/plain-node.yaml#L100 https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/snark-worker/templates/snark-coordinator.yaml#L114 https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/archive-node/values.yaml#L25 https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/archive-node/templates/archive.yaml#L117 https://github.com/MinaProtocol/mina/blob/f6958b429ae4eebc614ba74bd0a513ca58d3e769/helm/block-producer/templates/block-producer.yaml#L275

As explained in the Configuration Best Practices by Kubernetes themselves, we should try and avoid these hostPorts.