WIP: add inplace var mount

[raballew]

So, I did some research related to the systemd.volatile=overlay option in #2972 (comment)

An in-place mount of /var is pretty easy but it has unwanted side-effects. While for ostree another dependency at src/boot/ostree-prepare-root.service should be no issue, the problem is in ignition-ostree-mount-var.service which executes src/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.sh

set -euo pipefail

fatal() {
    echo "$@" >&2
    exit 1
}

if [ $# -ne 1 ] || { [[ $1 != mount ]] && [[ $1 != umount ]]; }; then
    fatal "Usage: $0 <mount|umount>"
fi

get_ostree_arg() {
    # yes, this doesn't account for spaces within args, e.g. myarg="my val", but
    # it still works for our purposes
    (
    IFS=$' '
    # shellcheck disable=SC2013
    for arg in $(cat /proc/cmdline); do
        if [[ $arg == ostree=* ]]; then
            echo "${arg#ostree=}"
        fi
    done
    )
}

do_mount() {
    ostree=$(get_ostree_arg)
    if [ -z "${ostree}" ]; then
        fatal "No ostree= kernel argument in /proc/cmdline"
    fi

    deployment_path=/sysroot/${ostree}
    if [ ! -L "${deployment_path}" ]; then
        fatal "${deployment_path} is not a symlink"
    fi

    stateroot_var_path=$(realpath "${deployment_path}/../../var")
    if [ ! -d "${stateroot_var_path}" ]; then
        fatal "${stateroot_var_path} is not a directory"
    fi

    echo "Mounting $stateroot_var_path"
    mount --bind "$stateroot_var_path" /sysroot/var
}

do_umount() {
    echo "Unmounting /sysroot/var"
    umount /sysroot/var
}

"do_$1"

During do_mount various checks are run which all fail if we do an inplace mount of /var somewhere else (such as /inplace/var/). So we have to tweak the setup in order to match the expectations of this unit by making /sysroot/${ostree} a symlink and ${deployment_path}/../../var a directory.

This is problematic because when ostree-prepare-root is run, ostree karg in /proc/cmdline evaluates to something different compared to the execution of ignition-ostree-mount-var.service.

• ostree-prepare-root: /sysroot//ostree/boot.1/fedora-coreos/
• ignition-ostree-mount-var: /sysroot//ostree/boot.1/fedora-coreos/12af9287ff3473ed4e23b911db71a706k1db71a706cc2d2c8a1aea8cb836763ad0ea9a5e15/0

Thus it is not possible to fake a directory structure that would pass the tests during ostree-prepare-root. From what I understand, this means I would need to modify the ignition-ostree-mount-var.service (or execute a dependency before running this service) but I do not understand the impact of changing this to something like:

#!/bin/bash
set -euo pipefail

fatal() {
    echo "$@" >&2
    exit 1
}

if [ $# -ne 1 ] || { [[ $1 != mount ]] && [[ $1 != umount ]]; }; then
    fatal "Usage: $0 <mount|umount>"
fi

do_mount() {
    echo "Mounting /sysroot/var"
    mount --bind /inplace/var /sysroot/var
}

do_umount() {
    echo "Unmounting /sysroot/var"
    umount /sysroot/var
}

"do_$1"

Additionally, building an image with this causes selinux to wrack havoc too. So running cosa run -c --kargs "enforcing=0 systemd.volatile=overlay" seems to be the only option actually being able to log in because otherwise I get:

boot.log

@alexlarsson @cgwalters wdyt, is adding another dependency a feasible approach or is it okay to modify ignition-ostree-mount-var.service (I am afraid that this might introduce a breaking change somewhere).

[openshift-ci]

Hi @raballew. Thanks for your PR.

I'm waiting for a ostreedev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.