Merge pull request #89 from desultory/dev

Add resume support, update plymouth support, improve mounts, simplify and expand tests, add basic bcachefs support

Author: desultory

.github/workflows/ubuntu.yml

@@ -15,16 +15,20 @@ permissions:
 jobs:
   build:
     runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.11', '3.12']
+
     steps:
     - uses: actions/checkout@v4
     - name: Obtain dependency projects
       run: |
         git clone https://github.com/desultory/zenlib
         git clone https://github.com/desultory/pycpio
-    - name: Set up Python 3.11
+    - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v5
       with:
-        python-version: "3.11"
+        python-version: ${{ matrix.python-version }}
     - name: Install system deps
       run: |
         sudo apt update
@@ -38,5 +42,5 @@ jobs:
         venv/bin/pip install .
     - name: Test fullauto.toml
       run: |
-        sudo venv/bin/python -m unittest discover tests
         sudo venv/bin/ugrd -d
+        sudo venv/bin/python -m unittest discover tests -v

docs/configuration.md

@@ -32,8 +32,9 @@ Modules write to a shared config dict that is accessible by other modules.
 
 ### base.core
 
-* `build_dir` (/tmp/initramfs) Defines where the build will take place.
-* `out_dir` (/tmp/initramfs_out) Defines where packed files will be placed.
+* `tmpdir` (/tmp) Sets the temporary directory as the base for the build and out dir.
+* `build_dir` (initramfs_build) If relative, it will be placed under `tmpdir`, defines the build directory.
+* `out_dir` (initramfs_out) If relative, it will be placed under `tmpdir`, defines the output directory.
 * `out_file` Sets the name of the output file, under `out_dir` unless a path is defined.
 * `clean` (true) forces the build dir to be cleaned on each run.
 * `hostonly` (true) Builds the initramfs for the current host, if disabled, validation is automatically disabled.
@@ -43,6 +44,57 @@ Modules write to a shared config dict that is accessible by other modules.
 * `binaries` - A list used to define programs to be pulled into the initrams. `which` is used to find the path of added entries, and `lddtree` is used to resolve dependendies.
 * `paths` - A list of directores to create in the `build_dir`. They do not need a leading `/`.
 
+#### Copying files
+
+Using the `dependencies` list will pull files into the initramfs using the same path on the host system.
+
+```
+dependencies = [ "/etc/ugrd/testfile" ]
+```
+
+#### Copying files to a different destination
+
+To copy files to a different path:
+
+```
+[copies.my_key]
+source = "/home/larry/.gnupg/pubkey.gpg"
+destination = "/etc/ugrd/pub.gpg"
+```
+
+#### symlink creation
+
+Symlinks are defined in the `symlinks` dict. Each entry must have a name, `source` and `target`:
+
+```
+[symlinks.pinentry]
+source = "/usr/bin/pinentry-tty"
+target = "/usr/bin/pinentry"
+```
+
+##### Device node creation
+
+Device nodes can be created by defining them in the `nodes` dict using the following keys:
+
+* `mode` (0o600) the device node, in octal.
+* `path` (/dev/node name) the path to create the node at.
+* `major` - Major value.
+* `minor` - Minor value.
+
+Example:
+
+```
+[nodes.console]
+mode = 0o644
+major = 5
+minor = 1
+```
+
+Creates `/dev/console` with permissions `0o644`
+
+> Using `mknod_cpio` from `ugrd.fs.cpio` will not create the device nodes in the build dir, but within the CPIO archive
+
+
 ### base.cmdline
 
 If used, this module will override the `mount_root` function and attempt to mount the root based on the passed cmdline parameters.
@@ -111,15 +163,28 @@ Similarly `ugrd.kmod.novideo` `nonetwork`, and `nosound` exist to ignore video,
 
 ### Filesystem modules
 
+`ugrd.fs.mounts` is the core of the filesystem module category and is included by default.
+
+Additional modules include:
+
+`ugrd.fs.btrfs` - Helps with multi-device BTRFS mounts, subvolume selection.
+`ugrd.fs.fakeudev` - Makes 'fake' udev entries for DM devices.
+`ugrd.fs.cpio` - Packs the build dir into a CPIO archive with PyCPIO.
+`ugrd.fs.livecd` - Assists in livecd image creation.
+`ugrd.fs.lvm` - Activates LVM mounts.
+`ugrd.fs.mdraid` - For MDRAID mounts.
+`ugrd.fs.resume` - Handles resume from hibernation.
+`ugrd.fs.test_image` - Creates a test rootfs for automated testing.
+
+#### ugrd.fs.mounts
+
 `autodetect_root` (true) Set the root mount parameter based on the current root label or uuid.
 `autodetect_root_dm` (true) Attempt to automatically configure virtual block devices such as LUKS/LVM/MDRAID.
 `autodetect_root_luks` (true) Attempt to automatically configure LUKS mounts for the root device.
 `autodetect_root_lvm` (true) Attempt to automatically configure LVM mounts for the root device.
 `autodetect_root_mdraid` (true) Attempt to automatically configure MDRAID mounts for the root device.
 `autodetect_init_mount'` (true) Automatically detect the mountpoint for the init binary, and add it to `late_mounts`.
 
-#### ugrd.fs.mounts
-
 `mounts`: A dictionary containing entries for mounts, with their associated config.
 
 `mounts.root` is predefined to have a destination of `/target_rootfs` and defines the root filesystem mount, used by `switch_root`.
@@ -193,55 +258,9 @@ Importing this module will run `btrfs device scan` and pull btrfs modules.
 * `root_subvol` - Set the desired root subvolume.
 * `_base_mount_path` (/root_base) Sets where the subvolume selector mounts the base filesytem to scan for subvolumes.
 
-#### symlink creation
-
-Symlinks are defined in the `symlinks` dict. Each entry must have a name, `source` and `target`:
-
-```
-[symlinks.pinentry]
-source = "/usr/bin/pinentry-tty"
-target = "/usr/bin/pinentry"
-```
+#### ugrd.fs.resume
 
-#### Copying files
-
-Using the `dependencies` list will pull files into the initramfs using the same path on the host system.
-
-```
-dependencies = [ "/etc/ugrd/testfile" ]
-```
-
-#### Copying files to a different destination
-
-To copy files to a different path:
-
-```
-[copies.my_key]
-source = "/home/larry/.gnupg/pubkey.gpg"
-destination = "/etc/ugrd/pub.gpg"
-```
-
-##### Device node creation
-
-Device nodes can be created by defining them in the `nodes` dict using the following keys:
-
-* `mode` (0o600) the device node, in octal.
-* `path` (/dev/node name) the path to create the node at.
-* `major` - Major value.
-* `minor` - Minor value.
-
-Example:
-
-```
-[nodes.console]
-mode = 0o644
-major = 5
-minor = 1
-```
-
-Creates `/dev/console` with permissions `0o644`
-
-> Using `mknod_cpio` from `ugrd.fs.cpio` will not create the device nodes in the build dir, but within the CPIO archive
+When enabled, attempts to resume after hibernation if resume= is passed on the kernel command line.
 
 ### Cryptographic modules
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ugrd"
-version = "1.25.0"
+version = "1.26.0"
 authors = [
   { name="Desultory", email="[email protected]" },
 ]

readme.md

@@ -56,6 +56,7 @@ The original goal of this project was to create an initramfs suitable for decryp
   - Hardlinks are automatically created for files with matching SHA256 hashes
   - Automatic xz compression
 * ZSH and BASH autocompletion for the `ugrd` command
+* Basic hibernation/resume support with `ugrd.fs.resume`
 * Similar usage/arguments as Dracut
 
 ## Support

src/ugrd/base/base.py

@@ -1,5 +1,5 @@
 __author__ = "desultory"
-__version__ = "5.1.1"
+__version__ = "5.1.3"
 
 from importlib.metadata import version
 from pathlib import Path
@@ -132,7 +132,7 @@ def rd_fail(self) -> list[str]:
     ]
     if "ugrd.base.plymouth" in self["modules"]:
         output += [
-            "    if check_var plymouth; then",
+            "    if plymouth --ping; then",
             '        plymouth display-message --text="Entering recovery shell"',
             "        plymouth hide-splash",
             "        bash -l",
@@ -189,7 +189,7 @@ def prompt_user(self) -> list[str]:
     output = ['prompt=${1:-"Press enter to continue."}']
     if "ugrd.base.plymouth" in self["modules"]:
         output += [
-            "if check_var plymouth; then",
+            "if plymouth --ping; then",
             '    plymouth display-message --text="$prompt"',
             "else",
             r'    echo -e "\e[1;35m *\e[0m $prompt"',
@@ -252,7 +252,18 @@ def edebug(self) -> list[str]:
 
 def einfo(self) -> list[str]:
     """Returns a bash function like einfo."""
-    return ["if check_var quiet; then", "    return", "fi", r'echo -e "\e[1;32m *\e[0m ${*}"']
+    if "ugrd.base.plymouth" in self["modules"]:
+        output = [
+            "if plymouth --ping; then",
+            '    plymouth display-message --text="${*}"',
+            "    return",
+            "fi",
+        ]
+    else:
+        output = []
+
+    output += ["if check_var quiet; then", "    return", "fi", r'echo -e "\e[1;32m *\e[0m ${*}"']
+    return output
 
 
 def ewarn(self) -> list[str]:
@@ -261,7 +272,7 @@ def ewarn(self) -> list[str]:
     """
     if "ugrd.base.plymouth" in self["modules"]:
         output = [
-            "if check_var plymouth; then",  # Always show the message if plymouth is running
+            "if plymouth --ping; then",  # Always show the message if plymouth is running
             '    plymouth display-message --text="Warning: ${*}"',
             "    return",  # Return early so echo doesn't leak
             "fi",
@@ -282,7 +293,7 @@ def eerror(self) -> str:
     """Returns a bash function like eerror."""
     if "ugrd.base.plymouth" in self["modules"]:
         return [
-            "if check_var plymouth; then",
+            "if plymouth --ping; then",
             '    plymouth display-message --text="Error: ${*}"',
             "    return",
             "fi",

src/ugrd/base/cmdline.py

@@ -1,5 +1,5 @@
 __author__ = "desultory"
-__version__ = "2.5.0"
+__version__ = "2.6.1"
 
 
 def parse_cmdline_bool(self) -> list[str]:
@@ -61,11 +61,20 @@ def mount_cmdline_root(self) -> list[str]:
 
 
 def export_exports(self) -> list[str]:
-    """Returns a bash script exporting all exports defined in the exports key."""
+    """Returns a bash script exporting all exports defined in the exports key.
+    Sets 'exported' to 1 once done.
+    If 'exported' is set, returns early.
+    """
     from importlib.metadata import PackageNotFoundError, version
 
     try:
         self["exports"]["VERSION"] = version(__package__.split(".")[0])
     except PackageNotFoundError:
         self["exports"]["VERSION"] = 9999
-    return [f'setvar {key} "{value}"' for key, value in self["exports"].items()]
+
+    check_lines = ["if check_var exported; then",
+                   '    edebug "Exports already set, skipping"',
+                   "    return", "fi"]
+    export_lines = [f'setvar "{key}" "{value}"' for key, value in self["exports"].items()]
+
+    return check_lines + export_lines + ["setvar exported 1"]

src/ugrd/base/core.py

@@ -1,5 +1,5 @@
 __author__ = "desultory"
-__version__ = "3.9.1"
+__version__ = "3.9.2"
 
 from pathlib import Path
 from typing import Union
@@ -172,7 +172,8 @@ def find_libgcc(self) -> None:
     try:
         ldconfig = self._run(["ldconfig", "-p"]).stdout.decode().split("\n")
     except RuntimeError:
-        return self.logger.critical("Unable to run ldconfig -p, if GCC is being used, this is fatal!")
+        self.logger.critical("Unable to run ldconfig -p, if GCC is being used, this is fatal!")
+        return self.logger.critical("This check can be disabled by setting `find_libgcc = false` in the configuration.")
 
     libgcc = [lib for lib in ldconfig if "libgcc_s" in lib and "(libc6," in lib][0]
     source_path = Path(libgcc.partition("=> ")[-1])
@@ -182,25 +183,37 @@ def find_libgcc(self) -> None:
     self["library_paths"] = str(source_path.parent)
 
 
-def _process_out_file(self, out_file):
-    """Processes the out_file configuration option."""
-    if Path(out_file).is_dir():
-        self.logger.info("Specified out_file is a directory, setting out_dir: %s" % out_file)
-        self["out_dir"] = out_file
+def _process_out_file(self, out_file: str) -> None:
+    """Processes the out_file.
+    If set to the current directory, resolves and sets the out_dir to the current directory.
+    If it starts with './', resolves it to the current directory.
+    If it is a directory, sets the out_dir to the directory.
+    If it's an absolute path, sets the out_dir to the parent directory.
+    """
+    out_file = str(out_file)
+    if out_file == "./" or out_file == ".":
+        current_dir = Path(".").resolve()
+        self.logger.info("Setting out_dir to current directory: %s" % current_dir)
+        self["out_dir"] = current_dir
         return
 
     if out_file.startswith("./"):
-        self.logger.debug("Relative out_file path detected: %s" % out_file)
-        self["out_dir"] = Path(".").resolve()
-        self.logger.info("Resolved out_dir to: %s" % self["out_dir"])
-        out_file = Path(out_file[2:])
-    elif Path(out_file).parent.is_dir() and str(Path(out_file).parent) != ".":
-        self["out_dir"] = Path(out_file).parent
-        self.logger.info("Resolved out_dir to: %s" % self["out_dir"])
-        out_file = Path(out_file).name
+        cwd = Path(".").resolve()
+        self.logger.info("Resolved relative out_file path: %s" % cwd)
+        out_file = cwd / out_file[2:]
     else:
         out_file = Path(out_file)
 
+    if out_file.is_dir():
+        self.logger.info("Specified out_file is a directory, setting out_dir: %s" % out_file)
+        self["out_dir"] = out_file
+        return
+
+    if out_file.parent.is_dir() and str(out_file.parent) != ".":
+        self["out_dir"] = out_file.parent
+        self.logger.info("Resolved out_dir to: %s" % self["out_dir"])
+        out_file = out_file.name
+
     self.data["out_file"] = out_file
 
 

src/ugrd/base/core.toml

@@ -1,7 +1,7 @@
 tmpdir = "/tmp"
 build_dir = "initramfs_build"
 _build_log_level = 10
-out_dir = "/tmp/initramfs_out"
+out_dir = "initramfs_out"
 clean = true
 find_libgcc = true
 hostonly = true