Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for QEMU rocker virtual switch device #2341

Open
matsievskiysv opened this issue Jan 25, 2024 · 1 comment
Open

Add support for QEMU rocker virtual switch device #2341

matsievskiysv opened this issue Jan 25, 2024 · 1 comment

Comments

@matsievskiysv
Copy link

QEMU rocker is an emulation of the switch ASIC with variable number of ports.

AFAIU from

"adapter_type": {
"description": "QEMU adapter type",
"type": "string",
"enum": ["e1000", "e1000-82544gc", "e1000-82545em", "e1000e", "i82550", "i82551", "i82557a", "i82557b", "i82557c", "i82558a",
"i82558b", "i82559a", "i82559b", "i82559c", "i82559er", "i82562", "i82801", "igb", "ne2k_pci", "pcnet", "rocker",
"rtl8139", "virtio", "virtio-net-pci", "vmxnet3"],
"default": "e1000"

it's currently treated as a regular network adapter, but it should be handled specially, because each rocker device may have up to 60 associated ports. Currently, rocker device cannot be used in GNS with more than one port, which negates its whole purpose.

Adding proper rocker device support would allow developers testing software aimed at switchdev based devices more easily.

An example of rocker device, connected to manually created taps

qemu-system-x86_64 \
     -drive if=ide,format=qcow2,file=disk.qcow2 \
     -boot c \
     -cpu host -M q35  \
     -enable-kvm \
     -m 2G,maxmem=4G \
     -nographic \
     -device '{"driver":"rocker","ports":["sw0p0","sw0p1","sw0p2","sw0p3"]}' \
     -netdev tap,id=sw0p0,ifname=rocker_tap0p0,script=no,downscript=no \
     -netdev tap,id=sw0p1,ifname=rocker_tap0p1,script=no,downscript=no \
     -netdev tap,id=sw0p2,ifname=rocker_tap0p2,script=no,downscript=no \
     -netdev tap,id=sw0p3,ifname=rocker_tap0p3,script=no,downscript=no

Here, the single rocker device is connected to four different taps.

Please, note, that due to this issue, QEMU CLI has been broken for some time and rocker could not be configured properly. The lowest QEMU version, that supports the JSON-like syntax above, is v8.2.

For the testing purposes, QEMU image with rocker driver could be created using the following commands

# install qemu >= 8.2.0
sudo apt install -t unstable qemu-system-x86

# build linux kernel deb package with rocker driver
sudo apt-get install build-essential bc kmod cpio flex libncurses-dev libelf-dev libssl-dev dwarves bison pahole
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.147.tar.xz -O linux.tar.xz
tar -xf linux.tar.xz && find . -maxdepth 1 -type d -name 'linux*' -exec mv {} linux \;
cp linux/arch/x86/configs/x86_64_defconfig rocker.conf
echo CONFIG_NET_SWITCHDEV=y >> rocker.conf
echo CONFIG_BRIDGE=y >> rocker.conf
echo CONFIG_ROCKER=m >> rocker.conf
make -C linux KCONFIG_ALLCONFIG=../rocker.conf alldefconfig
make -C linux -j$(nproc) deb-pkg
find . -maxdepth 1 -type f -name 'linux-image*.deb' -exec mv {} linux-image.deb \;


# build deps
sudo apt install parted debootstrap

# create virtual disk
qemu-img create -f qcow2 disk.qcow2 2G

# enter as root
su

export PATH=$PATH:/sbin:/usr/sbin
export TERM=xterm

# mount virtual disk
modprobe nbd
qemu-nbd --connect=/dev/nbd0 disk.qcow2 --cache=unsafe --discard=unmap

# partition disk
parted --align optimal --script --machine --fix /dev/nbd0 -- \
       mklabel gpt \
       mkpart fat32 '0%' 2MiB \
       set 1 bios_grub on \
       mkpart swap linux-swap 2MiB 100MiB \
       mkpart root ext4 100MiB '100%'
mkswap /dev/nbd0p2
mkfs.ext4 /dev/nbd0p3

# mount root
mkdir -p disk
mount /dev/nbd0p3 disk

# install system
debootstrap --include systemd,systemd-sysv \
            --variant=minbase \
            --arch amd64 bookworm ./disk

# mount special dirs
mount --bind /dev ./disk/dev
mount -t devpts /dev/pts ./disk/dev/pts
mount -t proc proc ./disk/proc
mount -t sysfs sysfs ./disk/sys
mount -t tmpfs tmpfs ./disk/tmp

# copy kernel package
cp -t disk/tmp/ linux-image.deb

# enter root
chroot ./disk

# set password
echo -e 'password\npassword' | passwd

# set hostname
echo rocker > /etc/hostname

# create fstab
UUID1=$(blkid /dev/nbd0p2 -s UUID -o value)
UUID2=$(blkid /dev/nbd0p3 -s UUID -o value)
echo "none /tmp tmpfs defaults 0 0" > /etc/fstab
echo "UUID=$UUID1 none swap sw 0 0" >> /etc/fstab
echo "UUID=$UUID2 / ext4 errors=remount-ro 0 1" >> /etc/fstab

# enable tty
systemctl enable [email protected]

# install kernel and bootloader
export DEBIAN_FRONTEND=noninteractive
export LANG="C.UTF-8"
apt-get update
mkdir -p /boot/grub
apt-get install -y grub2 initramfs-tools

dpkg -i /tmp/linux-image.deb

echo 'GRUB_DISABLE_OS_PROBER=true' >> /etc/default/grub
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="quiet"/GRUB_CMDLINE_DEFAULT=""/g' /etc/default/grub
sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="earlyprintk=serial,ttyS0 console=ttyS0"/g' /etc/default/grub
sed -i 's/#GRUB_TERMINAL=console/GRUB_TERMINAL=console/g' /etc/default/grub
grub-install --target=i386-pc --recheck /dev/nbd0
update-grub2

# install additional programs
apt-get install -y iproute2 inetutils-ping nano ethtool

# clear cache
apt-get clean
apt-get autoclean
rm -rf /var/lib/apt/lists/*

# autoprobe module
echo rocker > /etc/modprobe.d/00-rocker.conf

exit

# unmount all
for dev in tmp sys proc dev/pts dev; do
    umount ./disk/$dev
done

umount disk

rmdir disk
qemu-nbd --disconnect /dev/nbd0

# exit root
exit
@grossmj
Copy link
Member

grossmj commented Apr 23, 2024

This is interesting and would like to support this in a future version however there is some work involved to implement this. It will most likely happen after we release version 3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants