Implementing Kubernetes Operators with Python

Over the past few years, Kubernetes has become a default choice for container orchestration. The features and ease of use make it widely popular amongst the developers. Kubernetes API is one such feature that lets you query and manipulates the state of objects in Kubernetes. And to enhance the functionalities of K8s APIs, you can use a Kubernetes Operator. An Operator is an application-specific controller used to pack, deploy, and manage a Kubernetes application. It can replace the human Operator, someone with technical details of the application, understanding the business requirements, and working accordingly. For example, consider monitoring of the Kubernetes cluster using Prometheus Operator. If you do not use Operator then you need to setup Prometheus Kubernetes. It is a Kubernetes service discovery config and alert manager config for basic alerting rules like installing Grafana for visualization and many more. But when you use Prometheus Operator, everything is taken care of by the Operator. If you want some specific custom configurations or alerts, you need to pass your configuration to the Operator, and bingo, Operator makes it that simple! Just deploy an Operator, and you will find the complete monitoring stack implemented and provided with cluster insights. In simple words, an Operator is a process that runs in a pod and uses custom Kubernetes resources that do not exist in Kubernetes by default. And most importantly, the Operator communicates with the API server in automating the workflow of complicated applications. An Operator is basically a package of two components viz. Custom Resource Definition (CRD) and Controller and works on the principle of control loop Observe -> Diff -> Act. Observe the cluster, find out the difference between the current and desired state, and then act on it to bring it to the desired state. Your controller takes care of your custom resource defined in the cluster. It has the logic built inside it to handle the custom resource created by you. Let me explain this with an example. I will build an Operator in this article to understand the Operators in the easiest possible way. Here, I will create a Grafana operator that will create a Grafana instance and expose the Grafana dashboard to the end-user's port. For this, the end-user needs to create a Grafana object into the cluster and pass the desired nodeport where he wants to access the dashboard. Rest all the logic of creating a pod and nodeport service will be taken care by controller of your Operator. Let's see the actual process -

Writing Operator in Python:

You should use the Kubernetes Operator Pythonic Framework (Kopf) to build a simple Operator. Kopf is a framework used to build Kubernetes Operators in Python language. Just like any framework, Kopf provides you with both outer toolkit and inner libraries. The outer toolkit is used to run the Operator, to connect to the Kubernetes, and to collect the Kubernetes events into the pure Python functions of the Kopf-based Operator. The inner libraries assist with a limited set of common tasks of manipulating the Kubernetes objects. Let's check out the pre-requisites -

• Working Kubernetes cluster
• IDE for Python
• Docker Hub account

Once you have all these, here is how you can create an Operator -

• Create a Custom Resource Definition (CRD) An operator defines its custom resource definition. For our example, I will create a custom resource with the name Grafana with the Kubernetes CustomResourceDefinition object's help.

  $ cat < custom_resource_definition.yml 
  apiVersion: apiextensions.k8s.io/v1beta1 
  kind: CustomResourceDefinition 
  metadata: 
    name: grafana.opcito.org 
  spec: 
    scope: Namespaced 
    group: opcito.org 
    versions: 
      - name: v1 
        served: true 
        storage: true 
    names: 
      kind: Grafana 
      plural: grafana 
      singular: grafana 
      shortNames: 
        - gf 
        - gfn 
  EOF 
  $ kubectl apply -f custom_resource_definition.yml

  This will create a new object Grafana in the cluster, but it will need a controller to manage this object, which is our second step.

• Create an Operator handler or controller to manage the created CRD object The script given below will take care of the creation and deletion of your object:

  $ vi operator_handler.py 
  import kopf 
  import kubernetes 
  import yaml 
  @kopf.on.create('opcito.org', 'v1', 'grafana') 
  def create_fn(body, spec, **kwargs): 
  
      # Get info from grafana object 
      name = body['metadata']['name'] 
      namespace = body['metadata']['namespace'] 
      nodeport = spec['nodeport'] 
      image = 'grafana/grafana' 
      port = 3000 
      if not nodeport: 
          raise kopf.HandlerFatalError(f"Nodeport must be set. Got {nodeport}.") 
  
        # Pod template 
      pod = {'apiVersion': 'v1', 'metadata': {'name' : name, 'labels': {'app': 'grafana'}},'spec': {'containers': [ { 'image': image, 'name': name }]}} 
   
      # Service template 
      svc = {'apiVersion': 'v1', 'metadata': {'name' : name}, 'spec': { 'selector': {'app': 'grafana'}, 'type': 'NodePort', 'ports': [{ 'port': port, 'targetPort': port,  'nodePort': nodeport }]}} 
  
      # Make the Pod and Service the children of the grafana object 
      kopf.adopt(pod, owner=body) 
      kopf.adopt(svc, owner=body) 
    
      # Object used to communicate with the API Server 
      api = kubernetes.client.CoreV1Api() 
  
      # Create Pod 
      obj = api.create_namespaced_pod(namespace, pod) 
      print(f"Pod {obj.metadata.name} created") 
  
      # Create Service 
      obj = api.create_namespaced_service(namespace, svc) 
      print(f"NodePort Service {obj.metadata.name} created, exposing on port {obj.spec.ports[0].node_port}") 
  
      # Update status 
      msg = f"Pod and Service created for grafana object {name}" 
      return {'message': msg}
  @kopf.on.delete('opcito.org', 'v1', 'grafana') 
  def delete(body, **kwargs): 
      msg = f"Grafana {body['metadata']['name']} and its Pod / Service children deleted" 
      return {'message': msg} 
  

  The above script has comments mentioned on each line, which describes the task it is performing.

• Build Operator handler image The operator controller runs in a pod as a process, and hence we need to create a docker image for it using the below-mentioned dockerfile.

  $ vi Dockerfile 
  
  FROM python:3.7 
  RUN pip install kopf && pip install kubernetes 
  COPY operator_handler.py /operator_handler.py 
  CMD kopf run --standalone /operator_handler.py 
  

  Build the image and push it to the Docker Hub using the commands mentioned below:

  $ docker image build -t sanket07/operator-grafana:latest . 
  $ docker image push sanket07/operator-grafana:latest 
  

• Create a service account and role binding An operator needs permission to create resources in the cluster. I will assign a service account to the operator pod with permission to create resources in our cluster.

  $ cat < service_account.yml 
  apiVersion: v1 
  kind: ServiceAccount 
  metadata: 
    name: grafana-operator 
  EOF 
  
  $ kubectl apply -f service_account.yml 
    
  $ cat < service_account_binding.yml 
  apiVersion: rbac.authorization.k8s.io/v1 
  kind: ClusterRoleBinding 
  metadata: 
    name: grafana-operator 
  roleRef: 
    apiGroup: rbac.authorization.k8s.io 
    kind: ClusterRole 
    name: cluster-admin 
  subjects: 
    - kind: ServiceAccount 
      name: grafana-operator 
      namespace: default 
  
  EOF 
  
  $ kubectl apply -f service_account_binding.yml 
  

• Create deployment for operator in the cluster

  $ cat < grafana_operator.yml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
  name: grafana-operator
  spec:
  selector:
  matchLabels:
  app: grafana-operator
  template:
  metadata:
  labels:
  app: grafana-operator
  spec:
  serviceAccountName: grafana-operator
  containers:
  - image: sanket07/operator-grafana
  name: grafana-operator
  EOF
  
  $ kubectl apply -f grafana_operator.yml
  

  Verify if deployment for the operator is successfully running using the following command:

   $ Kubectl get pods 
  

Check for Grafana-operator pod:

Now that the Operator is successfully deployed, you should test it by creating a Grafana object in the cluster and then try to access the Grafana dashboard using node IP. You can specify the nodeport in the following object definition:

$ cat <<EOF > grafana.yml 
apiVersion: opcito.org/v1 
kind: Grafana 
metadata: 
  name: grafana-testing 
spec: 
  nodeport: 30087 
EOF 
$ kubectl apply -f grafana.yml 

You can check the created pod and the service for your Grafana object using the following command:

$ kubectl get pod,svc 

Verify pods and service presented with the object name:

Open the browser, visit the nodeIP and nodeport to check whether Grafana instance is available. You can log in to Grafana with admin:admin credentials. Now, this is an elementary example of creating a Grafana operator using Python. But the same process can be used to develop an operator to manage your complex applications and reduce the human intervention required for successful execution. So, try this method, and do not forget to share your experiences and queries in the comments section. Till then, stay safe and happy coding!