Implementing Kubernetes Operators with Python
Over the past few years, Kubernetes has become a default choice for container orchestration. The features and ease of use make it widely popular amongst developers. Kubernetes API is one such feature that lets you query and manipulates the state of objects in Kubernetes. And to enhance the functionalities of K8s APIs, you can use a Kubernetes Operator. An Operator is an application-specific controller used to pack, deploy, and manage a Kubernetes application. It can replace the human Operator, someone with technical details of the application, understanding the business requirements, and working accordingly. For example, consider monitoring the Kubernetes cluster using Prometheus Operator. If you do not use Operator, then you need to set up Prometheus Kubernetes. It is a Kubernetes service discovery config and alert manager config for basic alerting rules like installing Grafana for visualization and many more. But when you use Prometheus Operator, everything is taken care of by the Operator. If you want some specific custom configurations or alerts, you need to pass your configuration to the Operator, and bingo, the Operator makes it that simple! Just deploy an Operator, and you will find the complete monitoring stack implemented and provided with cluster insights. In simple words, an Operator is a process that runs in a pod and uses custom Kubernetes resources that do not exist in Kubernetes by default. And most importantly, the Operator communicates with the API server in automating the workflow of complicated applications. An Operator is basically a package of two components viz. Custom Resource Definition (CRD) and Controller and works on the principle of control loop Observe -> Diff -> Act. Observe the cluster, find out the difference between the current and desired state, and then act on it to bring it to the desired state. Your controller takes care of your custom resource defined in the cluster. It has the logic built inside it to handle the custom resource created by you. Let me explain this with an example. I will build an Operator in this article to understand the Operators in the easiest possible way. Here, I will create a Grafana operator that will create a Grafana instance and expose the Grafana dashboard to the end-user's port. For this, the end-user needs to create a Grafana object into the cluster and pass the desired nodeport where he wants to access the dashboard. Rest all the logic of creating a pod and nodeport service will be taken care of by the controller of your Operator. Let's see the actual process -
Writing Operator in Python:
You should use the Kubernetes Operator Pythonic Framework (Kopf) to build a simple Operator. Kopf is a framework used to build Kubernetes Operators in Python language. Just like any framework, Kopf provides you with both outer toolkit and inner libraries. The outer toolkit is used to run the Operator, connect to the Kubernetes, and collect the Kubernetes events into the pure Python functions of the Kopf-based Operator. The inner libraries assist with a limited set of common tasks of manipulating the Kubernetes objects. Let's check out the pre-requisites -
- Working Kubernetes cluster
- IDE for Python
- Docker Hub account
Once you have all these, here is how you can create an Operator -
- Create a Custom Resource Definition (CRD) An operator defines its custom resource definition. For our example, I will create a custom resource with the name Grafana with the Kubernetes CustomResourceDefinition object's help.
$ cat < custom_resource_definition.yml apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: grafana.opcito.org spec: scope: Namespaced group: opcito.org versions: - name: v1 served: true storage: true names: kind: Grafana plural: grafana singular: grafana shortNames: - gf - gfn EOF $ kubectl apply -f custom_resource_definition.yml
This will create a new object Grafana in the cluster, but it will need a controller to manage this object, which is our second step.
-
Create an Operator handler or controller to manage the created CRD object The script given below will take care of the creation and deletion of your object:
$ vi operator_handler.py import kopf import kubernetes import yaml @kopf.on.create('opcito.org', 'v1', 'grafana') def create_fn(body, spec, **kwargs): # Get info from grafana object name = body['metadata']['name'] namespace = body['metadata']['namespace'] nodeport = spec['nodeport'] image = 'grafana/grafana' port = 3000 if not nodeport: raise kopf.HandlerFatalError(f"Nodeport must be set. Got {nodeport}.") # Pod template pod = {'apiVersion': 'v1', 'metadata': {'name' : name, 'labels': {'app': 'grafana'}},'spec': {'containers': [ { 'image': image, 'name': name }]}} # Service template svc = {'apiVersion': 'v1', 'metadata': {'name' : name}, 'spec': { 'selector': {'app': 'grafana'}, 'type': 'NodePort', 'ports': [{ 'port': port, 'targetPort': port, 'nodePort': nodeport }]}} # Make the Pod and Service the children of the grafana object kopf.adopt(pod, owner=body) kopf.adopt(svc, owner=body) # Object used to communicate with the API Server api = kubernetes.client.CoreV1Api() # Create Pod obj = api.create_namespaced_pod(namespace, pod) print(f"Pod {obj.metadata.name} created") # Create Service obj = api.create_namespaced_service(namespace, svc) print(f"NodePort Service {obj.metadata.name} created, exposing on port {obj.spec.ports[0].node_port}") # Update status msg = f"Pod and Service created for grafana object {name}" return {'message': msg} @kopf.on.delete('opcito.org', 'v1', 'grafana') def delete(body, **kwargs): msg = f"Grafana {body['metadata']['name']} and its Pod / Service children deleted" return {'message': msg}
The above script has comments mentioned on each line, which describes the task it is performing.
-
Build Operator handler image The operator controller runs in a pod as a process, and hence we need to create a docker image for it using the below-mentioned dockerfile.
$ vi Dockerfile FROM python:3.7 RUN pip install kopf && pip install kubernetes COPY operator_handler.py /operator_handler.py CMD kopf run --standalone /operator_handler.py
Build the image and push it to the Docker Hub using the commands mentioned below:
$ docker image build -t sanket07/operator-grafana:latest . $ docker image push sanket07/operator-grafana:latest
-
Create a service account and role binding An operator needs permission to create resources in the cluster. I will assign a service account to the operator pod with permission to create resources in our cluster.
$ cat < service_account.yml apiVersion: v1 kind: ServiceAccount metadata: name: grafana-operator EOF $ kubectl apply -f service_account.yml $ cat < service_account_binding.yml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: grafana-operator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: grafana-operator namespace: default EOF $ kubectl apply -f service_account_binding.yml
-
Create deployment for the operator in the cluster
$ cat < grafana_operator.yml apiVersion: apps/v1 kind: Deployment metadata: name: grafana-operator spec: selector: matchLabels: app: grafana-operator template: metadata: labels: app: grafana-operator spec: serviceAccountName: grafana-operator containers: - image: sanket07/operator-grafana name: grafana-operator EOF $ kubectl apply -f grafana_operator.yml
Verify if deployment for the operator is successfully running using the following command:
$ Kubectl get pods
Check for Grafana-operator pod:
Now that the Operator is successfully deployed, you should test it by creating a Grafana object in the cluster and then try to access the Grafana dashboard using node IP. You can specify the nodeport in the following object definition:
$ cat <<EOF > grafana.yml apiVersion: opcito.org/v1 kind: Grafana metadata: name: grafana-testing spec: nodeport: 30087 EOF $ kubectl apply -f grafana.yml
You can check the created pod and the service for your Grafana object using the following command:
$ kubectl get pod,svc
Verify pods and service presented with the object name:
Open the browser, and visit the nodeIP and nodeport to check whether Grafana instance is available. You can log in to Grafana with admin:admin credentials. Now, this is an elementary example of creating a Grafana operator using Python. But the same process can be used to develop an operator to manage your complex applications and reduce the human intervention required for successful execution. So, try this method, and do not forget to share your experiences and queries in the comments section. Till then, stay safe and happy coding!