Manual setup of Kubernetes cluster on a Virtual Machine with kubeadm

How to setup Kubernetes cluster on a Debian Linux Virtual Machine

Kubernetes is one of the most popular container orchestrator. It is platform agnostic and although it uses Docker as its default container engine it can also run on Rocket developed by Core OS team. It is not the only orchestrator out there. Docker has Swarm mode integrated into it, so you can easily start orchestrating your containers without any additional software installed other then Docker on your host, but it is still in beta and it laks many functionalities that are in Kubenetes out of the box.
It is no wonder that leading Cloud providers such as AWS, Azure and GCE are racing to offer more and more functionalities on for container management and orchestration built on top of the Kubernetes itself.

Why installing Kubernetes manually on VM?

Although as mentioned, leading cloud providers are offering container orchestration out the box, configurable from their UI where you can configure your cluster with just few clicks, you might have to setup your cluster on premise depending on the company policy you are setting the cluster for. One more thing is a control. With on premise manual configuration you have full control over the cluster, but remember with power comes responsibility. Apart from full control over cluster, you will also become responsible for the uptime of the cluster, which on the cloud provider you do not have to think about. In the end it comes with it's pros and cones and it is up on you and your requirements whether you will host Kubernetes on premise or in cloud.

1. Install docker

Since latest version of kubelet service work with docker 17.03.0 you will have to specify this version to be installed through apt package manager, but let's first setup docker prerequisites.

First thing we need to do is to allow installing packages over https

sudo apt-get install \
     apt-transport-https \
     ca-certificates \
     curl \
     gnupg2 \
     software-properties-common
    

We need to add Docker GPG key to our VM

curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
    

To verify the GPG, you can just run the following command

sudo apt-key fingerprint 0EBFCD88
    

Next step is to add Docker repository for Debian Linux

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/debian \
   $(lsb_release -cs) \
   stable"
    

We are ready to tell apt to install but let's first update the packages on our system

sudo apt-get update
    

Unfortunately, latest Kubernetes version which is v1.11.2 does not work with latest version of Docker and for that reason we will have to specify the exact Docker version we want to install. For our case this is Docker 17.03.0. To get the exact name with of this version I will run the following command

apt-cache policy docker-ce
    

This will list all the version of docker-ce package for our Linux distro. Just pick the value from the list with version 17.03.0. In our case, since we are doing this demo on Linux Debian it will have the followng name which we'll pass to apt package manager to install

sudo apt-get install docker-ce=17.03.0~ce-0~debian-stretch
    

 This will install Docker 17.03.0 to our system, but since I already mentioned that Kubernetes does not work with other version of Docker, we'll tell apt not to update this package in case there is an update and update command is issued in the future

apt-mark hold docker-ce
    

Now we are ready to switch to Kubernetes installation.

2. Install Kubernetes and setting up the cluster

We will use kubeadm package to setup our Kubernetes cluster. At this time, latest version for kubeadm is v1.11.2, but you can always specify the version you want to install on your Debian host using same method as with docker-ce package.

Right now we are not going to specify version of kubeadm and will just install the latest version. As a first step we'll pull down GPG keys for the repository

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
    

Now we can update our packages and install kubeadm, kubectl and kubelet

apt-get update
apt-get install -y kubelet kubeadm kubectl
    

We will do the same thing as we did with docker-ce package and that is update locking. We'll tell apt not to update these three packages because we might cause issues due to compatibility with Docker

apt-mark hold docker-ce kubelet kubeadm kubectl
    

Before we initialize our cluster, we'll first pull down all the images needed for the Kubernetes, so that if we fail to initialize due to some issue, we can faster get the output

kubeadm config images pull
    

And finally, the momemt has tome for us to iitialize our cluster and spin up master of the Kubernetes cluster

kubeadm init
    

If everything goes well, you should see the output message with the join command so that you can join a new node to the cluster

If you wish to run your cluster as a normal user, with your user account you can copy the configuratin to your home directory and set environment variable

sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf  

echo "export KUBECONFIG=$HOME/admin.conf" >> /etc/profile
echo "export KUBECONFIG=$HOME/admin.conf" >> /etc/bash.bashrc
    

Otherwise, if you are fine running as su user, just set the environemt variables and persist them, so that your cluster is up in case of a reboot

export KUBECONFIG=/etc/kubernetes/admin.conf

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/bash.bashrc
    

Whohooo, you cluster is up and running...but!

Yep, this is not all. If you check your cluster nodes with the following command you will see that your node is not ready yet

kubectl get nodes
    

This is because you do not have the podnetwork setup yet. Podnetwork takes care of all the traffic inside the cluster, but luckily for us, kubeadm will point us to a url where we can get configuration yml files for our pod network. If you pay attention at output message after the init command you will notice this URL https://kubernetes.io/docs/concepts/cluster-administration/addons/. Just navigate to this URL and pick one of the options. I picked Weave Net and executed command from their website with kubectl 

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
    

Now if you check, your master node should show status ready

This should be all regarding the master setup, but to run the test we'll need at least one working node registered in the cluster, so next thing is configuring working node

3. Configure Kubernetes cluster node

Setting up worker node in the Kuberents cluster is pretty easy and 95% percent same as master setup. You will need same components and same configurations as for the master with one small difference.

You do not need to run kubeadm init command for the node. It is not a big deal if you already run it because you can always reset the initialized master node with this command

kubeadm reset --force
    

Now all you need is to join to a cluster by using the command you got once you initialize the cluster and master node. No worries if you lost the token join command, you can always issue a noew token from your master using kubeadm

sudo kubeadm token create --print-join-command 
    

Execute the output of this command on your worker node and you should have one worker node in cluster managed by the master node

kubectl get nodes
    

4. Test run

Now that we have Kubernetes cluster up and running let's spin up NGinx server and expose it outside. First we'll create a service which will be responsible for routing traffic to the pods marked with the same labels. We are going to do all this using yml files rather then explicitly run kubectl command with parameters

This will be our service yml file 

apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  labels:
    app: nginx
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
  selector:
    app: nginx
    

We'll save this file as nginx-svc.yml and run it against kubectl to create a new wervice from this yml declaration

kubectl create -f nginx-svc.yml
    

Now we'll have our service up and running and we can check it with simple kubectl command

kubectl get svc
    

We should see our service as ready

Next step is to deploy Nginx using deployment yml declaration file. This would be our Nginx deployment file content

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
    

We'll save this file as and nginx-dep.yml and create our Nginx deployment with kubectl command

kubectl create deployment -f nginx-dep.yml
    

And to verify our Nginx deployment we'll run one more command

kubectl get deployments
    

This will list our deployment and show the status

This is just the small scratch on the Kubernetes topic, but it should be a good start for you to begin your Kubernetes and Docker journey

References

• https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/
• https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/
• https://kubernetes.io/docs/concepts/cluster-administration/addons/#networking-and-network-policy
• https://docs.docker.com/install/linux/docker-ce/debian/

Disclaimer

Purpose of the code contained in snippets or available for download in this article is solely for learning and demo purposes. Author will not be held responsible for any failure or damages caused due to any other usage.

About the author

DEJAN STOJANOVIC

Dejan is a passionate Software Architect/Developer. He is highly experienced in .NET programming platform including ASP.NET MVC and WebApi. He likes working on new technologies and exciting challenging projects

CONNECT WITH DEJAN