While installing kubernetes on my on-prem lab , i did face strange issues on making them working. Thought of putting all of the steps together that made this work
Deployed 3 Ubuntu VM's
VM Name | Role | IP | Hostname |
master | control-plane,master | 10.109.xx.xx | master.nukescloud.com |
workerone | worker | 10.109.yy.yy | workerone.nukescloud.com |
workertwo | worker | 10.109.zz.zz | workertwo.nukescloud.com |
Ensured all three virtual machines are communicating between each other
Note : Below steps are to be performed on all of the three nodes
As a first step , Login as ‘sudo’ user because the following set of commands need to be executed with ‘sudo’ permissions. Then, update your ‘apt-get’ repository. Do this on all of the three nodes
We have to turn off the swap space because Kubernetes will start throwing random errors otherwise. After that you need to open the ‘fstab’ file and comment out the line which has mention of swap partition.
$ sudo su
# apt-get update
#swapoff -a
Second step , we need to install docker. Run the following commands
# sudo su
# apt-get update
# apt-get install -y docker.io
Third Step , we have to install these 3 essential components for setting up Kubernetes environment: kubeadm, kubectl, and kubelet
Run below commands before installing kubernetes environment
# apt-get update && apt-get install -y apt-transport-https curl
# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
# cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
# apt-get update
As Fourth Step, now its time to install the 3 essential components. kubelet is the lowest level component in Kubernetes. It’s responsible for what’s running on an individual machine. Kuebadm is used for administrating the Kubernetes cluster. Kubectl is used for controlling the configurations on various nodes inside the cluster.
apt-get install -y kubelet kubeadm kubectl
As Fifth step , we will change the configuration file of Kubernetes. Run the following command:
# nano /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
This will open a text editor, enter the following line after the last “Environment Variable”:
Environment="cgroup-driver=systemd/cgroup-driver=cgroupfs"
As Sixth Step , create a file daemon.json under /etc/docker
touch /etc/docker/daemon.json
vi /etc/docker/daemon.json
then paste this content in the json file
{
"exec-opts": ["native.cgroupdriver=systemd"]
}
Restart docker services
sudo systemctl enable docker && sudo systemctl daemon-reload && sudo systemctl restart docker
Seventh Step would be to initialize Kubernetes
# kubeadm init --apiserver-advertise-address=<ip-address-of-master-vm> --pod-network-cidr=10.244.0.0/16
Once initialized output would look like below
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 10.109.xx.xx:6443 --token jgzjjl.yyllcpqw5u2c9u9y \
--discovery-token-ca-cert-hash sha256:67feac133c1b3bda335a6179b99d5f88bf9cb3701ca7978fbc574d91d06abc92
Execute below commands as a non-root user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Execute below command as root user
export KUBECONFIG=/etc/kubernetes/admin.conf
Unless Pod Networking or CNI is installed , the core-dns pods will be in pending state
Eighth Step , Installing CNI. I've chosen flannel
Execute below command to install it
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
root@master:/home/osadmin# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
Ninth Step , Join nodes to cluster
root@workerone:/home/osadmin# kubeadm join 10.109.xx.xx:6443 --token jgzjjl.yyllcpqw5u2c9u9y --discovery-token-ca-cert-hash sha256:67feac133c1b3bda335a6179b99d5f88bf9cb3701ca7978fbc574d91d06abc92
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
root@workertwo:/home/osadmin# kubeadm join 10.109.46.53:6443 --token jgzjjl.yyllcpqw5u2c9u9y --discovery-token-ca-cert-hash sha256:67feac133c1b3bda335a6179b99d5f88bf9cb3701ca7978fbc574d91d06abc92
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
Last and Tenth Step is to list all the nodes and pods and see if they are ready
This concludes the kubernetes cluster installation and configuration
you missed some parts, like installing kubernetes and docker on worker nodes as well. Little edit required to this post other wise is good post. Try to post ContainerD with Kubernetes.