python-based-operator
python-based-operator copied to clipboard
A Kubernetes operator written in Python. It's meant as an introduction for those just starting out their journey writing operators.
A Python-Based k8s Operator
This project demonstrates how you can use plain Python to create a fully-functional k8s operator. To avoid re-inventing the wheel, Helm 3 is used internally by the operator to maintain the releases of the application it manages. However, if you want to customize this project to fit your needs and if your needs don't include Helm 3, you may safely remove that requirement from the code.
This operator can manage multiple instances of Prometheus. You can modify it to manage other types of applications if you wish. Prometheus was just chosen in this case because most engineers in the DevOps space are already familiar with it.
You instantiate Prometheus in a namespace by creating a PrometheusCluster custom resource in said namespace. A simple instance with defaults can be created via the following custom resource:
apiVersion: relaxdiego.com/v1alpha1
kind: PrometheusCluster
metadata:
name: simple-prometheus-instance
spec: {}
Demo
Watch it on YouTube.
Prior Art
Inspired by this Medium article
Dependencies
- Kubernetes 1.18 or higher
- Docker CE 17.06 or higher (For building the operator image)
- GNU Make
Optionally Use Microk8s
Use microk8s for testing this operator. It will make your life so much easier. Go on, I'll wait!
Once you have microk8s installed, run the following:
microk8s.enable dns rbac ingress registry storage
mkdir -p ~/.kube
microk8s.config > ~/.kube/config
kubectl cluster-info
Try It Out!
Build and Deploy the Operator
The following will build the image and deploy it in the python-based-operator
namespace.
make image deploy tag=localhost:32000/python-based-operator
NOTE: The address localhost:32000
is the address of the microk8s registry
addon that we enabled in the previous step. If you're not using microk8s,
just replace that address with either another registry address that you
have access to, or your Docker Hub username.
Create Your First Prometheus Cluster
There are sample PrometheusCluster files under the examples/
directory. After
deploying the operator, create a sample Prometheus cluster via kubectl:
kubectl create ns simple-prometheus-cluster
kubectl config set-context --current --namespace=simple-prometheus-cluster
kubectl apply -f examples/simple.yaml
Scale Up Your Prometheus Cluster
kubectl edit -f examples/simple.yaml
Go to the replicas:
field and change its value. Quit, save, then see your
number of prometheus pods scale accordingly.
Delete the Prometheus Cluster While Retaining its Data
Just run:
kubectl delete -f examples/simple.yaml
The volumes assocated with the pods will be retained and will be re-attached to the correct pod later on if you want to revive them.
Delete the Operator and Everything in the Example Namespace
kubectl delete -f examples/simple.yaml
make uninstall
kubectl delete ns simple-prometheus-cluster
Development Guide
Dependencies
- Python 3.8.x or higher
TIP: Make your life easier by using pyenv
Prepare Your Virtualenv (venv style)
python3 -m venv --prompt prometheus-operator .venv
source .venv/bin/activate
Prepare Your Virtualenv (pyenv-virtual style)
pyenv virtualenv 3.8.3 prometheus-operator-3.8.3
More on pyenv-virtualenv:
Install Development Dependencies
make dependencies
Add a Development Dependency
echo 'foo' >> src/requirements-dev.in
make dependencies
The src/requirements-dev.txt
file should now be updated and the foo
package installed in your local machine. Make sure to commit both files
to the repo to let your teammates know of the new dependency.
git add src/requirements-dev.*
git commit -m "Add foo to src/requirements-dev.txt"
git push origin
Add a Runtime Dependency
Add it to the install_requires
argument of the setup()
call in
src/setup.py
. For example:
setup(
name=_NAME,
version='0.1.0',
...
install_requires=[
'kubernetes>=11.0.0,<11.1.0',
'bar>=1.0.0,<2.0.0'
],
...
)
After having added the bar
dependency above, run the following:
make dependencies
The src/requirements.txt
file should now be updated and the bar package
installed in your local machine. Make sure to commit both files to the repo
to let your teammates know of the new dependency.
git add src/setup.py src/requirements.txt
git commit -m "Add bar as a runtime dependency"
git push origin
Run the Operator in Development Mode (Experimental)
This mode speeds up your development workflow by skipping the image creation
process, opting instead for to deploying it directly on your local machine.
To achieve a near-production runtime environment, we will create all the
resources in the k8s cluster except for the Deployment resource. Furthermore
this mode auto-generates a kubeconfig file from the Service Account we create
in the cluster so that the operator will still be constrained by the RBAC rules
that we specify under templates/rbac.yml
.
In order for Dev mode to work properly, we have to ensure that all runtime
dependencies are installed locally. This includes Helm 3.
Make sure to install that before proceeding. You do not need to manually install
the requirements in src/requirements.txt
since that will be done for you
automatically.
When all requirements are satisfied, go ahead and run:
make deploy-dev
You should see something like the following in the terminal:
python_based_operator.operator DEBUG Looking for credentials...
python_based_operator.operator DEBUG Loading from dev kube config
python_based_operator.operator DEBUG Loading CustomObjectsApi client
python_based_operator.operator INFO Watching prometheusclusters.relaxdiego.com/v1alpha1 events
If you need to make changes to the code, just press Ctrl-C
, edit the code,
then run make deploy-dev
again.
If you need something more streamlined than this, Okteto might be something of interest to you.
Force Re-Install Depedencies and Uninstall the Operator
Run the following
make reset
make dependencies
If you want something more thorough (and potentially destructive) than that, delete your virtual environment. Then start from the beginning of the Development Guide section.