One-Click Kubernetes Cluster Upgrade

Manually upgrading a cluster every time is always a pain — mindlessly repeating the same steps over and over serves no purpose. So I decided to script the upgrade process.

Before sharing my scripts, let me describe my cluster setup:

• 3 master nodes + 3 worker nodes
• Each node is created from the same VM template, all running CentOS.
• Each node is accessed via ssh key pair login

Upgrading Master Nodes

#!/bin/bash -e

is_first=true
for ip in $(kubectl get nodes --no-headers=true -o wide | grep master | awk '{print $6}');  
do
  node_name=$(kubectl get nodes --no-headers=true -o wide | grep ${ip} | awk '{print $1}')
  kubectl drain ${node_name} --ignore-daemonsets
  ssh -i "ssh/${node_name}" root@$ip 'yum update -y'
  if ${is_first};
  then
    ssh -i "ssh/${node_name}" root@$ip <<'ENDSSH'
      kubeadm upgrade plan
      kubeadm upgrade apply v1.19.4 -y
ENDSSH
    is_first=false
  else
     ssh -i "ssh/${node_name}" root@$ip 'kubeadm upgrade node'
  fi
  ssh -i "ssh/${node_name}" root@$ip <<-ENDSSH
    systemctl daemon-reload
    systemctl restart kubelet
ENDSSH
  kubectl uncordon ${node_name}
done

Upgrading Worker Nodes

#!/bin/bash -e

for ip in $(kubectl get nodes --no-headers=true -o wide | grep -v master | awk '{print $6}');  
do
  node_name=$(kubectl get nodes --no-headers=true -o wide | grep ${ip} | awk '{print $1}')
  kubectl drain ${node_name} --ignore-daemonsets --delete-local-data --force
  ssh -i "ssh/${node_name}" root@$ip  <<'ENDSSH' 
		yum update -y
    kubeadm upgrade node
    systemctl daemon-reload
    systemctl restart kubelet
ENDSSH
  kubectl uncordon ${node_name}
done

Q & A

Why not use Ansible? Because I haven’t used it before, and the logic here is simple enough.