High-Availability Kubernetes Cluster for Micro ONOS SDN Controller Deployment

Abstract

This repository provides a comprehensive guide for implementing a High-Availability (HA) Kubernetes cluster to deploy the micro ONOS SDN controller as microservices. Using Raspberry Pi devices as OpenFlow switches and Kubernetes workers, and virtual machines (via Vagrant and VirtualBox) for the control plane, this solution ensures robust and scalable deployment. The setup is automated with Ansible for ease of deployment and management. Follow the instructions below to configure and deploy the environment.

---

Resources

Control Plane and etcd:

• Master-1
• Master-2
• Master-3

Worker Nodes:

• Raspberry Pi 4 (ARM) x1
• Raspberry Pi 4 (ARM) x2
• Raspberry Pi 4 (ARM) x3

Container Network Interface (CNI):

• Calico

HA Kubernetes Cluster:

• Keepalived
• HAproxy

---

1. Install Raspbian on Each Node

Preparing an SD Card (Linux)

1. Download Raspbian:
   Get the latest Raspbian image from the official Raspberry Pi site:
   Download Raspbian

2. Write the Image to the SD Card:

   sudo dd if=YYYY-MM-DD-raspios-buster-arm64-lite.img of=/dev/sdX bs=16M status=progress

3. Provision Wi-Fi Settings on First Boot: Update the bootstrap/wpa_supplicant.conf file:

   ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
   update_config=1
   country=AU
   
   network={
       ssid="your_wifi_ssid"
       psk="your_wifi_password"
       key_mgmt=WPA-PSK
   }
   

   Copy the file to the SD card:

   cp bootstrap/wpa_supplicant.conf /mnt/boot/

4. Enable SSH:

   cp bootstrap/ssh /mnt/boot/ssh

5. Example Commands:

   sudo umount /media/<user>/boot
   sudo dd if=2022-09-22-raspios-bullseye-arm64.img of=/dev/<disk> bs=16M status=progress
   sync

---

2. Update Cluster Configuration

• Edit cluster.yml to match your setup.
• Configure static IPs via DHCP or manual assignment.

---

3. Install Required Tools

Install sshpass:

sudo apt-get install sshpass

Install Kubernetes:

ansible-playbook -i cluster.yml playbooks/upgrade.yml

Overclock Raspberry Pi (Optional):

ansible-playbook -i cluster.yml playbooks/overclock-rpis.yml

Install k3s on the Cluster:

ansible-playbook -i cluster.yml site.yml

---

4. Set Up High-Availability Kubernetes Cluster

Install Keepalived and HAproxy:

sudo apt-get install keepalived haproxy -y

Configure HAproxy:

Edit /etc/haproxy/haproxy.cfg:

global
    log /dev/log local0
    chroot /var/lib/haproxy
    pidfile /var/run/haproxy.pid
    maxconn 4000
    user haproxy
    group haproxy
    daemon

frontend kube-apiserver
    bind *:6443
    mode tcp
    default_backend kube-apiserver

backend kube-apiserver
    mode tcp
    balance roundrobin
    server master1 192.168.1.101:6443 check
    server master2 192.168.1.102:6443 check
    server master3 192.168.1.103:6443 check

Restart HAproxy:

sudo systemctl restart haproxy

Configure Keepalived:

Edit /etc/keepalived/keepalived.conf:

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    authentication {
        auth_type PASS
        auth_pass mypassword
    }
    virtual_ipaddress {
        192.168.1.250
    }
}

Restart Keepalived:

sudo systemctl restart keepalived

---

References

• Install K3s on Raspberry Pi with Ansible
• Vagrant Get Started
• Set Up an HA Kubernetes Cluster