Skip to content

Advanced Configuration Features

Jump to bottom
Stefan Lankes edited this page Jun 23, 2024 · 14 revisions

Kernel configuration

The library operating system HermitOS provides following features, which can be configured implicitly by the configuration of hermit-sys:

  • smp: By selecting the smp feature, the kernel is able to run on a symmetric multiprocessing system, otherwise it is only working on a single-processor system.
  • pci: Enable the support of PCI device. This feature must be enabled, if the library operating system is running on Qemu.
  • pci-ids: Enable the support of readable PCI IDs, which produce readable debug messages.
  • acpi: By selecting the acpi feature, basic ACPI will be activated.
  • fsgsbase: FSGSBASE is an Intel Architecture extension that allows applications to directly write to the FS / GS segment registers. This allows fast switching to different threads in HermitOS. Please select fsgsbase to improve the performance if the processor provides this extension.
  • vga: By selecting the via feature, the kernel uses also a video graphic card to show the results.
  • tcp: The TCP/IP stack smoltcp will be integrated if the feature smoltcp will be selected.
  • dhcpv4: Per default, DHCPv4 isn't supported. If the features dhcpv4 and smoltcp will be selected, HermitOS is able to use smoltcp's DHCPv4 client.

For instance, the following configuration of hermit-sys activates the TCP/IP stack with DHCPv4 support for PCI-based architecture:

[target.'cfg(all(any(target_arch = "x86_64", target_arch = "aarch64"), target_os = "hermit"))'.dependencies.hermit-sys]
features = ["pci", "tcp", "dhcpv4"]

By default, only smp, fsgsbase, pci, pci-ids and acpi is activated.

Using common cargo commands to build and run HermitOS applications

If the configuration file .cargo/config is created in the project folder as follows, it is possible to build HermitOS with common cargo commands.

[unstable]
build-std = ["std", "core", "alloc", "panic_abort"]
build-std-features = ["compiler-builtins-mem", "compiler-builtins-asm"]

[build]
target = "x86_64-unknown-hermit"

[target.x86_64-unknown-hermit]
runner = "uhyve -v"

Now, the default target is defined as x86_64-unknown-hermit, which represents HemitOS Consequently, the command cargo run --bin rusty_demo builds a HermitOS application and start it within in uhyve.

Controlling kernel message verbosity

HermitOS uses the lightweight logging crate log to print kernel messages. If the environment variable HERMIT_LOG_LEVEL_FILTER is set at compile time to a string matching the name of a LevelFilter, then that value is used for the LevelFilter. If the environment variable is not set, or the name doesn't match, then LevelFilter::Info is used by default, which is the same as it was before.

For instance, the following command build HermitOS with debug messages:

$ HERMIT_LOG_LEVEL_FILTER=Debug cargo build -Z build-std=std,core,alloc,panic_abort --target x86_64-unknown-hermit

Network support

By a persistent tap interface

One option to enable ethernet support is to setup a tap device on the host system. For instance, the following command establish the tap device tap10 on Linux:

$ sudo ip tuntap add tap10 mode tap
$ sudo ip addr add 10.0.5.1/24 broadcast 10.0.5.255 dev tap10
$ sudo ip link set dev tap10 up
$ sudo bash -c 'echo 1 > /proc/sys/net/ipv4/conf/tap10/proxy_arp'

Add the feature smoltcp in the Cargo.toml. This includes the network stack smoltcp and offers TCP/UDP communication.

# Cargo.toml

[target.'cfg(target_os = "hermit")'.dependencies]
hermit-sys = "0.2.*"
default-features = false
features = ["pci","tcp"]

Per default, HermitOS's network interface uses 10.0.5.3 as IP address, 10.0.5.1 for the gateway and 255.255.255.0 as network mask. The default configuration could be overloaded at compile time by the environment variables HERMIT_IP, HERMIT_GATEWAY and HERMIT_MASK. For instance, the following command sets the IP address to 10.0.5.100.

$ HERMIT_IP="10.0.5.100" cargo build -Z build-std=std,core,alloc,panic_abort --target x86_64-unknown-hermit

Per default, Hermit's network interface uses 9.9.9.9 (Quad9) and 1.1.1.1 (Cloudflare) as DNS servers. The default configuration could be overloaded at compile time by the environment variables HERMIT_DNS1 and HERMIT_DNS2. For instance, the following command sets the DNS servers to 8.8.8.8 and 8.8.4.4.

$ HERMIT_DNS1="8.8.8.8" HERMIT_DNS2="8.8.4.4" cargo build -Z build-std=std,core,alloc,panic_abort --target x86_64-unknown-hermit --features hermit/dns,hermit/tcp

Currently, HermitOS does only support network interfaces through virtio. To use it, you have to start HermitOS in Qemu with following command:

$ qemu-system-x86_64 -cpu qemu64,apic,fsgsbase,rdtscp,xsave,fxsr \
        -enable-kvm -display none -smp 1 -m 1G -serial stdio \
        -device isa-debug-exit,iobase=0xf4,iosize=0x04 \
        -kernel path_to_loader/hermit-loader-x86_64 \
        -initrd path_to_app/app \
        -netdev tap,id=net0,ifname=tap10,script=no,downscript=no,vhost=on \
        -device virtio-net-pci,netdev=net0,disable-legacy=on

DHCP support

If DHCP support is required, the support must be activated in hermit-sys. Add the feature smoltcp and dhcpv4 in the file Cargo.toml. This includes the network stack smoltcp and enables DHCPv4 support.

# Cargo.toml

[target.'cfg(target_os = "hermit")'.dependencies]
hermit-sys = "0.2.*"
default-features = false
features = ["pci", "tcp", "dhcpv4"]

In this case, the usage of Qemu's user networking is possible. For the example, the small example web server can be started with following command:

$ qemu-system-x86_64 -smp 1 -m 256M -device isa-debug-exit,iobase=0xf4,iosize=0x04 -kernel lpath_to_loader/hermit-loader-x86_64 -initrd path_to_hhtpd/httpd -display none -serial stdio -netdev user,id=u1,hostfwd=tcp::9975-:9975 -device rtl8139,netdev=u1 --enable-kvm -cpu host

In case of the usage of Qemu's user networking, the network interface RTL 8139 must be used. The command already open with hostfwd=tcp::9975-:9975 the port of the web server. In this case, Qemu is listening on the local port 9975 and forward all requests to the guest server, which is also listening at port 9975.

By adding -object filter-dump,id=f1,netdev=u1,file=dump.dat to the command line, the network traffic can be captured and stored in the file dump.dat. Afterwards, the traffic can be analyzed by Wirehshark. For instance, the following command dump the network traffic in a readable text form.

tshark -r ./dump.dat

Using VirtioFS to share a file system (only required when using QEMU)

It is possible to share the host file system with HermitOS when using Qemu, but you must use QEMU with VirtioFS support to do this.

Now, we use virtiofsd, the virtio-fs vhost-user device daemon written in Rust.

Building virtiofsd

  1. Install dependencies: The virtiofsd project depends on libcap-ng and libseccomp. Install these dependencies either by building them from their sources, or by installing the packages from package manager: dnf install libcap-ng-devel libseccomp-devel on Fedora/CentOS/RHEL or apt install libcap-ng-dev libseccomp-dev on Debian/Ubuntu.
  2. Download the source code from the virtiofsd project: git clone https://gitlab.com/virtio-fs/virtiofsd.git
  3. cd virtiofsd
  4. Compile virtiofsd using cargo: cargo build --release
  5. The binary can be found at /target/release
  6. (optional) Add the location of the binary to PATH or put the binary to the directory for executable programs (e.g., /usr/local/bin).

Run Unikernel with VirtioFS support

  1. Prepare a filesystem for the guest. For instance, create a directory within /tmp, which will serve as the mountable directory within the guest, e.g., execute mkdir /tmp/guestfs
  2. Start the VirtioFS daemon and export the current working directory to the guest: virtiofsd --socket-path=/tmp/vhostqemu --shared-dir=/tmp/guestfs

In a new terminal (Do not close the previous terminal!):

  1. Give non-root-users access to the socket: sudo chmod 777 /tmp/vhostqemu
  2. Start the application with VirtioFS, e.g., the following will execute the HermitOS demo:
qemu-system-x86_64 -cpu host \
    -enable-kvm \
    -display none \
    -smp 1 \
    -m 256M \
    -serial stdio \
    -kernel path_to_loader/hermit-loader-x86_64 \
    -initrd hermit-rs/target/x86_64-unknown-hermit/release/rusty_demo \
    -chardev socket,id=char0,path=/tmp/vhostqemu \
    -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=root \
    -object memory-backend-file,id=mem,size=256M,mem-path=/dev/shm,share=on \
    -numa node,memdev=mem

These commands mount the host directory /tmp/guestfs in the guest as /root. Per default /root is the working directory of HermitOS. The working directory can changed by setting the environment variable HERMIT_WD before compiling HermitOS. In addition, the tag root in argument list of Qemu specifies the path within the guest.

Note: The demo application will try to read /etc/hostname, which still fails because the directory etc isn't mounted within the quest.

Enabling DAX support

DAX mapping allows HermitOS direct access to file content from the host file cache.

The device section of the Qemu command line must be changed as follows: qemu-system-x86_64 -enable-kvm -cpu host -display none -smp 1 -m 256M -serial stdio -kernel path_to_loader/hermit-loader-x86_64 -initrd target/x86_64-unknown-hermit/debug/rusty_demo -chardev socket,id=char0,path=/tmp/vhostqemu -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=root,cache-size=256M -object memory-backend-file,id=mem,size=256M,mem-path=/dev/shm,share=on -numa node,memdev=mem

Clone this wiki locally