HermitOS on K8S

With the support of KubeVirt, RustyHermit is able to run on Kubernetes, also known as K8S. The following steps show how to deploy our small web server on a K8S cluster.

At first, we have to build the server. The support of DHCP must be activated and also the activation of VGA is useful to debug log messages. Consequently, httpd should be created with the command cargo build --manifest-path examples/httpd/Cargo.toml --no-default-features --features pci,acpi,smoltcp,vga,dhcpv4.

Typically KubeVirt uses Qemu as hypervisor and a boot image for Qemu must be created.

• Creation of a file, which will represent the virtual disk. With the command dd if=/dev/zero of=disk.img bs=1k count=1M, a file will be created with a size of 1GB.
• Creation of a boot partition: echo ',,,*;' | sfdisk disk.img
• Formating of the virtual disk: mkfs.ext2 -F -E offset=1048576 disk.img
• Create device maps from partition tables: kpartx -a -v disk.img
• kpartx creates typically the device /dev/loop0. If loop0 is already in use, kpartx is able to use another number. This depends on your local system. After the creation of the loop device, the device is mountable by mount /dev/mapper/loop0p1 /mnt/
• Installing the boot loader grub: grub-install --root-directory=/mnt --locales= --themes= --fonts= --no-floppy --modules="normal part_msdos ext2 multiboot biosdisk" loop0
• Copy the RustyHermit application to the boot directory of the disk: cp target/x86_64-unknown-hermit/release/httpd /mnt/boot/
• Copy the boot loader to the boot directory: cp loader/target/x86_64-unknown-hermit-loader/release/rusty-loader /mnt/boot/loader
• Creation of a grub configuration file at /mnt/boot/grub/grub.cfg with following content:

default=0
timeout=0
menuentry "httpd" {
    multiboot --quirk-bad-kludge /boot/loader
    module /boot/httpd
    boot
}

• Unmount virtual disk: umount /mnt
• Detach loop device: kpartx -d disk.img
• Convert disk image to the qcow2 format: qemu-img convert -f raw -O qcow2 disk.img disk.qcow2

This disk image must be provided to all containers, which are orchestrated by K8S. The easiest way to it, is the usage of a container registry. KubeVirt provides a standardized base wrapper container image that serves up a user provided virtual disk. Thus, the virtual disk image must be pushed into the container registry. The base wrapper container will serve up any virtual disk placed in the /disk directory as a block device consumable of the VM.

To create a container registry, the following Dockerfile must be created:

FROM scratch
ADD disk.qcow2 /disk/

disk.qcow2 represents the path to the disk image, which is created due the previous steps. With the command docker build -f Dockerfile ., the container will be created and must be uploaded to the container registry. The uploaded process depends on the preferred container registry.

To register a service as K8S, the following sketch of a K8S configuration file can be used. In principle, the file specifies the virtual machine and a port, where the service is listening. The last line specifies the path to the container registry and must be replaced by the path to the preferred registry.

apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachineInstance
metadata:
  name: httpd
  spec:
    terminationGracePeriodSeconds: 0
   domain:
     cpu:
       cores: 1
       model: qemu64
     resources:
       requests:
         memory: 64M
     devices:
       disks:
       - name: containerdisk
         disk: {}
       interfaces:
       - name: default
         bridge: {} # connect through a bridge
         model: rtl8139
         ports:
         - name: http
           port: 9975
           protocol: TCP
   networks:
   - name: default
     pod: {}
   volumes:
   - name: containerdisk
     containerDisk:
       image: url_to_the_container_registry:latest

If you want to see a real-life example, take a look at the configuration of the GitLab CI Pipeline, which builds the container disk and deploys the service with the configuration file httpd.yml.