SLATE for the lightweight edge with k3s

Posted on Tuesday, December 10 2019.

Kubernetes (K8s) is a powerful container orchestration tool. k3s is a lightweight distribution of Kubernetes that strips away a number of features while remaining fully compliant with up-stream Kubernetes. It allows easier deployment when compared to kubeadm and all in a binary less than 40MB. k3s is a fantastic solution for deploying Kubernetes with SLATE on smaller devices, older hardware, and even IOT. In this blog post, we explain how a k3s and SLATE can provide a tidy, lightweight edge federation.

k3s is an open-source Kubernetes distribution provided by Rancher Labs. It is easy to install and uses half the memory of a standard Kubernetes installation. This is useful for many environments where resources are limited. As more users have been exploring k3s and leveraging edge computing for applications in different domains including science and research, we thought we’d try it out with our SLATE platform. This will provide a way to make the SLATE platform more accessible to institutions who are interested in leveraging SLATE for lightweight applications, such as the perfSONAR testpoint.

First, let’s go through an installation of k3s. We’ll start with a virtual machine which we’ve given a public IP address. Many of the following commands need to be run as superuser, so it is probably a good idea to sudo su - before beginning.
Our virtual machine came with a fresh installation of CentOS 7, so we’ll assume that here.

Prepare the operating system

Run the operating system requirements steps in the SLATE cluster installation docs:

#Install yum utils
yum install yum-utils -y

#Disable SELinux
setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

#Disable swap
swapoff -a
sed -e '/swap/s/^/#/g' -i /etc/fstab

#Disable firewalld
systemctl disable --now firewalld

#Disable root login over ssh
sed -i --follow-symlinks 's/#PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config

#Use iptables for Bridged Network Traffic
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

Install Docker

Install Docker. We recommend using the Docker CE runtime with SLATE and k3s:

# Add Docker stable repo to Yum
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# Install the latest version of DockerCE and container 
yum install docker-ce docker-ce-cli containerd.io -y

# Enable Docker on reboot through systemctl
systemctl enable --now docker

Run k3s

Run the k3s install script:

curl -sfL https://get.k3s.io | sh -

Enable kubeconfig for kubectl access

Time to KubeConfig. To enable kubectl access for root, copy the k3s yaml file to $HOME/.kube/config:

mkdir -p $HOME/.kube
cp -f /etc/rancher/k3s/k3s.yaml $HOME/.kube/config	
chown $(id -u):$(id -g) $HOME/.kube/config

To enable kubeconfig for a single session instead run

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Modify the kubeconfig for SLATE

Make sure the server’s address in the kube config file is pointing to a public IP address. Refer to the sample file below to see where the public IP address needs to be added

Here is a sample k3s yaml file:

apiVersion: v1
clusters:
- cluster:
   certificate-authority-data: '<data redacted>'
   server: https://localhost:6443 # <- Change localhost to your public IP or DNS name
 name: kubernetes
contexts:
- context:
   cluster: kubernetes
   user: kubernetes-admin
 name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
 user:
   client-certificate-data: '<data redacted>'
   client-key-data: '<data redacted>'

Check for a master k3s node by running:

kubectl get nodes

The following step is not required for k3s as the master node is already untainted when k3s is installed:

//kubectl taint nodes k3d-k3s-default

Deploy pod network (Calico)

Download calico.yaml file using

wget https://docs.projectcalico.org/v3.8/manifests/calico.yaml

Update CALICO_IPV4POOL_CIDR value on line 625 of the calico.yaml file to k3s server default ip for example 10.42.0.0/16. Now Apply calico.yaml file by running:

kubectl apply -f calico.yaml

Deploy the load balancer (MetalLB)

Follow the Load Balancer part of the SLATE cluster installation document and apply the metallb.yaml file to your cluster and create the ConfigMap for MetalLB to your cluster after changing the IP address range in the metallb-config.yaml file.

Install the SLATE client

Install SLATE in your k3s cluster by following the SLATE installation instructions.

Register with the federation

Create the k3s SLATE cluster by running a command similar to the one provided below. Change the name of the cluster, group, org:

./slate cluster create umich-k3s-test --group slate-dev-testing2 --org SLATE

In the above example the cluster name is umich-k3s-test.

Deploy an application

Deploy an application to your k3s SLATE cluster by following Deploying an Application part in the SLATE CLI docs.

Uninstall k3s

If you need to uninstall k3s, run this command:

/usr/local/bin/k3s-uninstall.sh

Summary

In summary, we were able to successfully set up a k3s cluster in a virtual environment and deploy a SLATE application to the k3s cluster in less than an hour.

The SLATE Team