Add support for EKS-D

[richardcase]

trafficstars

/kind feature

Describe the solution you'd like I would like to be able to use EKS-D for my EC2 based (i.e. no-EKS) cluster.

Anything else you would like to add: See this blog post for an intro to EKS-D.

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

[richardcase]

/lifecycle frozen

[randomvariable]

CC @kschumy

We probably don't want to build EKS-D images that are hosted by the Kubernetes SIG Cluster Lifecycle if they're carrying patches which are not part of a released version of upstream Kubernetes, in the same way we don't host images for OpenShift or VMware Tanzu Kubernetes Grid. However, we could support automated lookup of AWS hosted AMIs from a known location for EKS.

[richardcase]

I did some playing around with this a while back and used ClusterResourceSets to install things CSI:

apiVersion: cluster.x-k8s.io/v1alpha3
kind: Cluster
metadata:
  name: "capi-eksd-test"
  labels:
    eksd: "true"
    cni: "calico"
spec:
  clusterNetwork:
    pods:
      cidrBlocks: ["192.168.0.0/16"]
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
    kind: AWSCluster
    name: "capi-eksd-test"
  controlPlaneRef:
    kind: KubeadmControlPlane
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
    name: "capi-eksd-test-control-plane"

And then in the KubeadmControlPlane i specified eksd images from dockerhub. There was an issue with the pause image tag (which has now been fixed) so i copied them to my account:

kind: KubeadmControlPlane
apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
metadata:
  name: "capi-eksd-test-control-plane"
spec:
  replicas: 1
  infrastructureTemplate:
    kind: AWSMachineTemplate
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
    name: "capi-eksd-test-control-plane"
  kubeadmConfigSpec:
    initConfiguration:
      nodeRegistration:
        name: '{{ ds.meta_data.local_hostname }}'
        kubeletExtraArgs:
          cloud-provider: aws
    clusterConfiguration:
      kubernetesVersion: "v1.18.9-eks-1-18-1"
      imageRepository: "registry.hub.docker.com/richardcase"
      etcd:
        local:
          imageRepository: "registry.hub.docker.com/richardcase"
          imageTag: "v3.4.14-eks-1-18-1"
      dns:
        imageRepository: "registry.hub.docker.com/richardcase"
        imageTag: "v1.7.0-eks-1-18-1"
      apiServer:
        extraArgs:
          cloud-provider: aws
      controllerManager:
        extraArgs:
          cloud-provider: aws
    joinConfiguration:
      nodeRegistration:
        name: '{{ ds.meta_data.local_hostname }}'
        kubeletExtraArgs:
          cloud-provider: aws
    preKubeadmCommands:
      - 'sh /tmp/eksd.sh'
    files:
    - owner: root:root
      path: /tmp/eksd.sh
      permissions: "0700"
      content: |
        #!/bin/bash
        echo "Installing curl"
        apt-get install curl

        echo "Downloading EKSD binaries"
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/plugins/v0.8.7/cni-plugins-linux-amd64-v0.8.7.tar.gz -o cni.tar.gz
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/kubernetes/v1.18.9/kubernetes-client-linux-amd64.tar.gz -o client.tar.gz
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/kubernetes/v1.18.9/kubernetes-server-linux-amd64.tar.gz -o server.tar.gz
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/kubernetes/v1.18.9/kubernetes-node-linux-amd64.tar.gz -o node.tar.gz
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/kubernetes/v1.18.9/bin/linux/amd64/kubectl -o kubectl
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/kubernetes/v1.18.9/bin/linux/amd64/kubelet -o kubelet
        curl -L https://distro.eks.amazonaws.com/kubernetes-1-18/releases/1/artifacts/aws-iam-authenticator/v0.5.2/aws-iam-authenticator-linux-amd64-v0.5.2.tar.gz -o aws-iam-auth.tar.gz

        chmod +x ./kubelet
        chmod +x ./kubectl

        echo "Installing CNI"
        mkdir  /tmp/cni
        tar xvfz cni.tar.gz -C /tmp/cni
        rm -rf /tmp/cni/LICENSES
        rm /tmp/cni/ATTRIBUTION.txt
        cp /tmp/cni/* /opt/cni/bin/

        echo "Installing server components"
        mkdir /tmp/server
        tar xvfz server.tar.gz -C /tmp/server
        cp /tmp/server/kubernetes/server/bin/kubeadm /usr/bin/
        cp /tmp/server/kubernetes/server/bin/kubelet /usr/bin/
  version: "v1.18.9"

[richardcase]

I will update the sample yaml now the pause image is tagged correctly to the AWS images directly.

[randomvariable]

/priority backlog

[richardcase]

/remove-lifecycle frozen

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[richardcase]

We could cover this with a template

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.