Skip to main content

Kubernetes: 9. Taints & Tolerations

Taints and Tolerations

  • Taints and Toleration are used to describe what pods can be scheduled on what node
  • They wont force a pod to schedule on the node that has a certain taint that cant be tolerated by a pod
  • If a pod has to be mandatorily scheduled on a particular node then node-affinity has to be used
Taints
  • Taints are set on the nodes
  • By default none of the pods have any toleration set
  • So if a node is tainted, then with default settings on the pods, none of the pods can be scheduled on the tainted node
  • Note that by default the scheduler never schedules a pod on a master node
  • This is because there is a taint set on the master node
  • This can be modified if required, but as a best practice only the management software is supposed to be run on the master node
kubectl describe node <node> | grep Taint

Tolerations

  • Tolerations are set on the pods
How to set taint on a node? Use kubectl as described below

kubectl taint node <node-name> <taint-key-value-pairs>:<taint-effect> 
-> Create a taint on a node

kubectl taint node node01 app=blue:NoSchedule 
-> Create a taint on node01 with app=blue &taint-effect to NoSchedule

kubectl taint nodes node01 app:NoSchedule- 
-> Remove the taint on a node

Taint Effect
  1. NoSchedule
    • This means if the pod cannot tolerate the taint, then it will not be scheduled on the node
    • Any existing pods on the node that cannot tolerate the taint on the node are NOT EVICTED
  2. PreferNoSchedule
    • In this case Kubernetes will try not to schedule the pod on the node that has a taint that pod cannot tolerate but it will not guarantee
  3. NoExecute
    • In this case Kubernetes will not schedule any new pod on the node that has a taint that pod cannot tolerate
    • If there is any existing pod that cannot tolerate the taint, then it will be evicted out
How to set toleration on a pod? Update pod-definition file as described below
  • Add toleration section in the spec section
  • Add key, operator, value and effect keys and their corresponding values
  • These values are likely to be matched with the taint of the node on which pod has to run
  • Note that the values have to put in double-quotes. This is mandatory
pod-definition.yaml
apiVersion: v1
kind: Pod
metadata:
    name: myapp-pod
    labels:
        app: myapp
        location: IN


spec:
    containers:
    - name: nginx-container
      image: nginx

    tolerations:
    - key: "app"
      operator: "Equal"
      value: "blue"
      effect: "NoSchedule"

Example of how taints and tolerations works:
kubectl taint nodes node1 key1=value1:NoSchedule
kubectl taint nodes node1 key1=value1:NoExecute
kubectl taint nodes node1 key2=value2:NoSchedule

----------
tolerations:
- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoSchedule"

- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoExecute"
  • In this case, the pod will not be able to schedule onto the node, because there is no toleration matching the third taint.
  • But an existing pod will continue to run if it is already running on the node when the taint is added,
  • This is because the third taint is the only one of the three that is not tolerated by the pod.
  • Normally, if a taint with effect NoExecute is added to a node, then any pods that do not tolerate the taint will be evicted immediately,
  • Pods that do tolerate the taint will never be evicted.
  • However, a toleration with NoExecute effect can specify an optional tolerationSeconds field that dictates how long the pod will stay bound to the node after the taint is added.
Labels:

Comments

Post a Comment

Popular posts from this blog

Kubernetes: 19. Configure Application

Configuring application consists of Configuring commands and arguments on applications Configuring environment variables Configuring secrets Docker Commands docker run ubuntu  -> Runs ubuntu container and exit, container CMD is set to [bash], so the container quitely exits docker run ubuntu echo "Hello World" -> Runs ubuntu container, prints "Hello World" exits quitely. To update the default settings, create your own image from the base image lets call this ubuntu-sleeper image FROM ubuntu CMD sleep 5 CMD can also be mentioned in the JSON format like CMD ["sleep", "5"] Note that with JSON format the first element should always be the command to execute,  for eg, it CANNOT be ["sleep 5"] Run build the new ubuntu-sleeper image and run the new image docker build -t ubuntu-sleeper .  -> Build the image docker run ubuntu-sleeper -> Run the new image So the new image will launch ubuntu container, sleep for 5 seconds and quitely ex...
Post a Comment
Read more

Kubernetes: 15. Multiple Schedulers

Custom Scheduler Kubernetes allows to create custom schedulers There can be multiple schedulers running at a same time apart from the default scheduler or A custom scheduler can replace the default kube-scheduler to become the default one So a few pods that requires additional checks apart from taints and toleration, node affinity can go through the custom scheduler before getting scheduled on the node Whereas the rest of the pods can go through the default kube-scheduler Create Custom Scheduler We can either download the kube-scheduler and run it as a service or alternatively create it using a static pod Below here we are downloading the binaries to run it The property scheduler-name is used to define the name of the scheduler, if not set then it will be defaulted to default-scheduler For your custom schedulers, update this property name to set a custom name for your scheduler For Static pods, the name can be updated directly in the pod-definition file Use kubectl create -f <pod-de...
Post a Comment
Read more

Kubernetes: 21. Secrets

Passwords In the webapps we store the properties file for storing and retrieving the data required by application But we never store the application passwords, truststore, keystore passwords etc here We might store them in an encrypted format, but storing them as plain text is not the correct way In Kubernetes we store these sensitive information in Secrets https://medium.com/avmconsulting-blog/secrets-management-in-kubernetes-378cbf8171d0 Secrets Secrets are used to store the sensitive information They are similar to ConfigMaps, except that they are stored in hashed or encoded format Note that they are only encoded (using base64) but are not encrypted So secrets are a safe option to store sensitive information but infact they are not the safest option As such secret objects should be not checked into source code tools, its best to store them encrypted at REST in ETCD Again as in ConfigMaps, we have to create the secrets object first and then inject them into the pods There are 2 ways ...
Post a Comment
Read more