k8s-infra
k8s-infra copied to clipboard
Information to bootstrap vms using dedicated server, local machine and setup using Ansible Playbooks
= Automating the deployment of a kubernetes/ocp cluster Snowdrop Team (Charles Moulliard, Antonio Costa) :icons: font :revdate: {docdate} :toc: left :toclevels: 3 ifdef::env-github[] :tip-caption: :bulb: :note-caption: :information_source: :important-caption: :heavy_exclamation_mark: :caution-caption: :fire: :warning-caption: :warning: endif::[]
== Introduction
[.lead]
This project details the prerequisites
and steps
necessary to automate the installation of a Kubernetes (aka k8s) cluster or Openshift 4 top of one of the following cloud provider:
- Red Hat https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/[OpenStack] (RHOS-PSI)
- Red Hat https://github.com/resource-hub-dev[ResourceHub] - Work in progress
- IBM Cloud
- Hetzner
- Kind**
NOTE: kind is not a cloud provider but a tool able to run a k8s cluster on a container engine
== Before you start
[WARNING]
All the commands mentioned on this project are to be executed at the root folder of the repository, except if stated otherwise.
== Prerequisites
This project uses Ansible. Check the link:ansible/README.adoc[Ansible Document] for the installation and usage instructions.
To use the scripts, playbooks, part of this project, some prerequisites are needed. It is not mandatory to install all of them and the following chapters will mention which ones are needed.
- https://kind.sigs.k8s.io/docs/user/quick-start/#installation[kind]
- https://docs.docker.com/engine/install/[Docker] or https://podman.io/docs/installation[podman]
- https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-and-upgrading-ansible[Ansible]. Version >= 2.14
- https://www.python.org/downloads/[Python]. Version >= 3.11
- https://www.passwordstore.org/[passwordstore]
- https://github.com/hetznercloud/cli[hcloud] (optional)
=== Python
Several requirements are provided as Python libraries, including Ansible, and are identified on the link:requirements.txt[] file.
Using a Python Virtual Environment is recommended and can be created using the following command:
[source,bash]
python3 -m venv .snowdrop-venv
After creating the virtual environment start using it with the following command:
[source,bash]
source .snowdrop-venv/bin/activate
The venv will be in use when the (.snowdrop-venv)
prefix is shown on the bash prompt.
The python requirements can be installed by executing:
[source,bash]
pip3 install -r requirements.txt
[NOTE]
For more information check the link:ansible/README.adoc#python-venv[Python Virtual Env] section on our Ansible README.
=== Ansible
Several Ansible Galaxy collections are used as part of this project and are listed in the link:collections/requirements.yml[] file. To install them execute the following command.
[source,bash]
ansible-galaxy collection install -r ./collections/requirements.yml --upgrade
=== Kind
Tools: docker (or podman), kind
To automate the installation of a k8s "kind" cluster locally like also to set up an ingress controller or a docker container registry, use our opinionated xref:kind/README.adoc[bash scripts] :-).
You can find more information about kind tool using the official documentation - https://kind.sigs.k8s.io/docs/user/quick-start/
=== Minikube
Tools: docker (or podman), minikube
See the https://kubernetes.io/docs/tasks/tools/install-minikube/[official documentation] to install minikube
on Macos, Linux or Windows
== Cloud provider
The provisioning process towards the cloud providers relies on the following assumptions:
- Password store is installed/configured and needed k/v created
- Flavor, volume, capacity (cpu/ram/volume) and OS can be mapped with the playbook of the target cloud provider
- Permissions have been set to allow to provision a VM top of the target cloud provider
- SSH key exist and has been imported (or could be created during provisioning process)
and will include the following basic steps:
- Create a VM, mount a volume and import the SSH key
- Execute a pos installation script to install some needed services
- Register the Hostnames against the domain name (using Lets'encrypt and DNS provider)
- Deploy an ocp4 cluster and configure the different ingress routes to access the console, API, registry, etc
NOTE: Optionally, we could also install different kubernetes tools if we would like to access/use the VM (e.g. kubectl, oc, helm, k9s, konfig, ect - see tooling section).
=== Red Hat
This section details how to provision an Openshift 4 cluster using one of Red Hat environments available such as:
==== OpenStack - RHOS PSI
Tools: password store, ansible
The link:openstack/README.adoc[OpenStack] page explains the process using the RHOS cloud provider.
==== https://resourcehub.redhat.com/[Resource Hub]
Tools: password store, ansible
Work in progress
=== IBM Cloud
Tools: password store, ansible
See xref:ibm-cloud/README.adoc[ibm-cloud]
=== Hetzner
==== Bare metal
Tools: password store, ansible, hcloud
See xref:hetzner/README.adoc[hetzner] page explaining how to create a vm.
==== Virtualized machine
Tools: password store, ansible, hcloud
See xref:hetzner/README-cloud.adoc[hetzner-cloud] page explaining how to create a cloud vm.
== Cluster Deployment
As the vm is now running and the docker daemon is up, you can install your k8s
distribution using either one of the following approaches :
=== Kubernetes
You can then use the following instructions to install a Kubernetes cluster with the help of Ansible and the xref:doc/k8s.adoc[roles we created]
=== OpenShift
- Simple using the
oc
binary tool and the command https://github.com/openshift/origin/blob/master/docs/cluster_up_down.md[oc cluster up] within the vm - More elaborated using
Ansible
tool and one of the following playbook/role: **oc cluster up
xref:doc/oc.adoc[role] **openshift-ansible
all-in-one playbook as described xref:doc/cloud.adoc[here]
== Sandbox
xref:sandbox/sandbox.adoc[Material] not actively maintained to create a VM, run on your desktop a k8s cluster or provision it with Istio, Jaeger, Fabric8 launcher, Ansible Broker catalog, etc