Kubernetes on 64 Bit OS Raspberry Pi 4

So my obsession of getting k8s running in a practical small scale lab continues. I took a dive on “Installing Ubuntu 19.10 on Raspberry Pi” because I wanted to upgrade to a 64-bit OS. Ubuntu gave this to me out of the box with some slight configurations. From this I took the four original Pi and upgraded each of them to Ubuntu 19.10.

The original install of k8s I did with Docker, and at the time of this post Docker CE support of Ubuntu 19.10 was not there. So in a deep dive of research I found Canonical’s Microk8s. This looked to be a better alternative for this experiment. So with that let’s get some Microk8s installed and running.

Again we are starting from this post “Installing Ubuntu 19.10 on Raspberry Pi”, if you need the list of things to buy for this “Kubernetes Running on Raspberry Pi”. One more thing I have added a few more steps on security, I recommend you do them, but you don’t have to.

Enabling CGroups

So we know that you need to update the Pi’s to enable cgroups that was outlined here in the documentation from Canonical. But their directions are for Raspberry Pi’s 2 and 3. If you notice the OS is different as well, where they are using Ubuntu 18.04 and not the 19.10 version. That being said the place where you enable has changed.

sudo nano /boot/firmware/nobtcmd.txt

Add the following to the end of this file:

cgroup_enable=memory cgroup_memory=1

You will need to reboot so:

sudo reboot

SSH

I took some time to actually do SSH correctly, well, in the context of creating a home lab. Leaving SSH open with just a password to log in allows people to attempt to brute force it. So I below will push a key locally to your Pi:

ssh-copy-id ubuntu@192.168.1.x # note the x here is supposed to be a number I masked it in my example

It will ask for the password you set for your Pi.

From here you can SSH in to your Pi with this command:

ssh ubuntu@192.168.1.x # note the x here is supposed to be a number I masked it in my example

If you set up your key correctly you will be prompted to enter your passphrase. Next we are going to remove the ability to SSH in with just your password.

sudo nano /etc/ssh/sshd_config

Find this in the file:

PasswordAuthentication yes

Change it to:

PasswordAuthentication no

Full disclaimer, I am not an expert on this subject. I did in my set up and configuration suggest you do as well. For more information on this topic please check this out.

Installing Microk8s

On each of my Pis I ran the following command.

sudo snap install microk8s --classic --channel=1.16/stable

Note that the channel at the time of this post is 1.16/stable.

If you attempt to check the status:

microk8s.status

You will get the following message:

Insufficient permissions to access _MicroK8s_.
You can either try again with sudo or add the user ubuntu to the __microk8s__ group:

    sudo usermod -a -G microk8s ubuntu

The new group will be available on the user's next login.

So do the following commands:

sudo usermod -a -G microk8s ubuntu
logout

You will have to SSH back in to the Pi then kick off the following command:

microk8s.start

This will take a few moments to start everything up. Next let’s enable dns:

microk8s.enable dns

Lastly we will enable storage:

microk8s.enable storage

Joining them together

So at this point you should have Microk8s installed on each of your Pis. SSH in to each of them. On the Pi that is going to be your master in the cluster run the following command:

microk8s.add-node

You will get something like this:

Join node with: microk8s.join 192.168.1.50:25000/QemrXHozDmVQdq...

If the node you are adding is not reachable through the default interface you can use one of the following:
 microk8s.join 192.168.1.50:25000/QemrXHozDmVQdq...
 microk8s.join 10.1.67.0:25000/QemrXHozDmVQdq...

On the next Pi run the following from that print out:

microk8s.join 192.168.1.50:25000/QemrXHozDmVQdq...

NOTE!!! You will have to do this microk8s.add-node each time you add a node, the token is unique.

At this point once you have done this for each of your Pis you can run the following command:

microk8s.kubectl get node

And get the following:

NAME              STATUS     ROLES    AGE     VERSION
192.168.1.5x      Ready      <none>   24h     v1.16.3
192.168.1.5x      Ready      <none>   24h     v1.16.3
192.168.1.5x      Ready      <none>   24h     v1.16.3
pi-64-master-0x   Ready      <none>   26h     v1.16.3

So, there it is Microk8s on four Raspberry Pis. Next I will take a dive on adding some features to this such as RabbitMQ.