Kubernetes: Scheduling notes

Manual Scheduling

If scheduler pod is not present in kube-system, add nodeName in spec to manually schedule a pod

Labels and Selectors

Standard method to group things together
in pod-definition file

apiVersion: v1
kind: Pod
metadata:
  name: nginx
	labels:
		app: App1
		function: frontend
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2

Now one can select them using:

kubectl get pods --selector app=App1

Kubernetes uses labels and selectors to connect different type of objects together

Get multiple labels

kubectl get pods --selector app=App1,tier=frontend

Taints and Tolerations

They are used to set restrictions on what pods can be scheduled on a node.

Nodes have taints while pods have tolerances

Taints

kubvectl taint nodes node-name key=value:taint-effect

3 taint effects:

NoSchedule: Pod will not be scheduled
PreferNoSchedule: Will try not placing pod in the node but that’s not guaranteed
NoExecute: New pods will not be scheduled on the node and existing pods will be evicted if they do not tolerate the taint.

Example:

kubectl taint nodes node1 app=blue:NoSchedule

Tolerations

apiVersion:
kind: Pod
metadata:
	name: myapp-pod
spec:
	containers:
	- name: nginx-container
		image: nginx
	tolerations:
	- key: "app"
		operator: "Equal"
		value: "blue"
		effect: "NoSchedule"

Node Selectors

apiVersion: v1
kind: Pod
metadata:
  name: bee
spec:
  containers:
  - image: nginx
    name: bee
  nodeSelector:
    size: large

Resource Requests

Default CPU: 0.5 and Memory: 256 Mi

Setting Default memory

apiVersion: v1
kind: LimitRange
metadata:
  name: mem-limit-range
spec:
  limits:
  - default:
      memory: 512Mi
    defaultRequest:
      memory: 256Mi
    type: Container

Setting default CPU

apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-limit-range
spec:
  limits:
  - default:
      cpu: 1
    defaultRequest:
      cpu: 0.5
    type: Container

To change, update YAML

apiVersion: v1
kind: Pod
metadata:
  name: bee
spec:
  containers:
  - image: nginx
    name: bee
  resources:
    requests:
      memory: "1Gi"
      cpu: 1

Resource Limits

apiVersion: v1
kind: Pod
metadata:
  name: bee
spec:
  containers:
  - image: nginx
    name: bee
  resources:
    requests:
      memory: "1Gi"
      cpu: 1
    limits:
      memory: "2Gi"
      cpu: 2

Daemon Sets

1 instance of pod is available on every node

Example: kube-proxy

Can be used for agent based nodes such as for monitoring solution

YAML Definition

apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: elasticsearch
  name: elasticsearch
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: elasticsearch
    spec:
      containers:
      - image: k8s.gcr.io/fluentd-elasticsearch:1.20
        name: fluentd-elasticsearch

Static Pods

Static Pods are managed directly by the kubelet daemon on a specific node, without the API server observing them. Unlike Pods that are managed by the control plane (for example, a Deployment ); instead, the kubelet watches each static Pod (and restarts it if it fails).

Static Pods are always bound to one Kubelet on a specific node.

The kubelet automatically tries to create a mirror Pod on the Kubernetes API server for each static Pod. This means that the Pods running on a node are visible on the API server, but cannot be controlled from there. The Pod names will be suffixed with the node hostname with a leading hyphen.

Additional Scheduler - kubeadm

Deploy scheduler as a pod