Kubernetes: 15. Multiple Schedulers

Custom Scheduler

• Kubernetes allows to create custom schedulers
• There can be multiple schedulers running at a same time apart from the default scheduler or
• A custom scheduler can replace the default kube-scheduler to become the default one
• So a few pods that requires additional checks apart from taints and toleration, node affinity can go through the custom scheduler before getting scheduled on the node
• Whereas the rest of the pods can go through the default kube-scheduler

Create Custom Scheduler

• We can either download the kube-scheduler and run it as a service or alternatively create it using a static pod
• Below here we are downloading the binaries to run it
• The property scheduler-name is used to define the name of the scheduler, if not set then it will be defaulted to default-scheduler
• For your custom schedulers, update this property name to set a custom name for your scheduler
• For Static pods, the name can be updated directly in the pod-definition file
• Use kubectl create -f <pod-definition-file> to create the custom scheduler
• Run kubectl get pods -n kube-system to view the custom scheduler
• Note that when there are multiple master nodes in the cluster, then there will be multiple versions of scheduler running
• Only one version of scheduler should be active at a time to make sure pods are scheduled correctly
• For this there is a property "leader-elect" this must be set to true so that a leader is elected during HA (multi master) set up
• When you add the custom scheduler, add the property "lock-object-name=<scheduler-name>" so that the scheduler knows this is a custom scheduler

Configure Pod to use custom scheduler

• To configure the pod to use the custom scheduler, add the schedulerName property in the spec section
• When the pod is created it will use the custom scheduler for scheduling
• If the scheduler is not configured correctly then the pod remains in PENDING state

pod-definition.yaml

apiVersion: v1

kind: Pod

metadata:

    name: myapp-pod

    labels:

        app: myapp

spec:

    containers:

    - name: nginx-container

      image: nginx

    schedulerName: my-custom-scheduler

View Events

• To confirm that the pod picked the custom scheduler view the events and verify that custom scheduler is used
• Alternatively look for the scheduler logs in the kube-system namespace
• Remember that schedulers run kube-system namespace

kubectl get events

-> Shows the events run in the current namespace

kubectl logs scheduler <custom-scheduler> -n kube-system

-> Logs of the custom scheduler