SLATE for the lightweight edge with k3s
Kubernetes (K8s) is a powerful container orchestration tool. k3s is a lightweight distribution of Kubernetes that strips away a number of features while remaining fully compliant with up-stream Kubernetes. It allows easier deployment when compared to kubeadm and all in a binary less than 40MB. k3s is a fantastic solution for deploying Kubernetes with SLATE on smaller devices, older hardware, and even IOT. In this blog post, we explain how a k3s and SLATE can provide a tidy, lightweight edge federation.
k3s is an open-source Kubernetes distribution provided by Rancher Labs. It is easy to install and uses half the memory of a standard Kubernetes installation. This is useful for many environments where resources are limited. As more users have been exploring k3s and leveraging edge computing for applications in different domains including science and research, we thought we’d try it out with our SLATE platform. This will provide a way to make the SLATE platform more accessible to institutions who are interested in leveraging SLATE for lightweight applications, such as the perfSONAR testpoint.
First, let’s go through an installation of k3s. We’ll start with a virtual machine which we’ve given a public IP address. Many of the following commands need to be run as superuser, so it is probably a good idea to
sudo su - before beginning.
Our virtual machine came with a fresh installation of CentOS 7, so we’ll assume that here.
Prepare the operating system
Run the operating system requirements steps in the SLATE cluster installation docs:
#Install yum utils
yum install yum-utils -y
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
sed -e '/swap/s/^/#/g' -i /etc/fstab
systemctl disable --now firewalld
#Disable root login over ssh
sed -i --follow-symlinks 's/#PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
#Use iptables for Bridged Network Traffic
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
Install Docker. We recommend using the Docker CE runtime with SLATE and k3s:
# Add Docker stable repo to Yum
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# Install the latest version of DockerCE and container
yum install docker-ce docker-ce-cli containerd.io -y
# Enable Docker on reboot through systemctl
systemctl enable --now docker
Run the k3s install script:
curl -sfL https://get.k3s.io | sh -
Enable kubeconfig for kubectl access
Time to KubeConfig. To enable kubectl access for root, copy the k3s yaml file to $HOME/.kube/config:
mkdir -p $HOME/.kube
cp -f /etc/rancher/k3s/k3s.yaml $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
To enable kubeconfig for a single session instead run
Modify the kubeconfig for SLATE
Make sure the server’s address in the kube config file is pointing to a public IP address. Refer to the sample file below to see where the public IP address needs to be added
Here is a sample k3s yaml file:
certificate-authority-data: '<data redacted>'
server: https://localhost:6443 # <- Change localhost to your public IP or DNS name
- name: kubernetes-admin
client-certificate-data: '<data redacted>'
client-key-data: '<data redacted>'
Check for a master k3s node by running:
kubectl get nodes
The following step is not required for k3s as the master node is already untainted when k3s is installed:
//kubectl taint nodes k3d-k3s-default
Deploy pod network (Calico)
Download calico.yaml file using
Update CALICO_IPV4POOL_CIDR value on line 625 of the calico.yaml file to k3s server default ip for example 10.42.0.0/16. Now Apply calico.yaml file by running:
kubectl apply -f calico.yaml
Deploy the load balancer (MetalLB)
Follow the Load Balancer part of the SLATE cluster installation document and apply the metallb.yaml file to your cluster and create the ConfigMap for MetalLB to your cluster after changing the IP address range in the metallb-config.yaml file.
Install the SLATE client
Install SLATE in your k3s cluster by following the SLATE installation instructions.
Register with the federation
Create the k3s SLATE cluster by running a command similar to the one provided below. Change the name of the cluster, group, org:
./slate cluster create umich-k3s-test --group slate-dev-testing2 --org SLATE
In the above example the cluster name is umich-k3s-test.
Deploy an application
Deploy an application to your k3s SLATE cluster by following Deploying an Application part in the SLATE CLI docs.
If you need to uninstall k3s, run this command:
In summary, we were able to successfully set up a k3s cluster in a virtual environment and deploy a SLATE application to the k3s cluster in less than an hour.