Skip to content

Commit 5c52b25

Browse files
cgwalterscgwalters
committed
docs: Add comprehensive ostree-ext container storage documentation
Document how container images are stored as ostree commits, including: container/mod.rs: - On-disk storage structure (ref namespace, layer storage, merge commit) - Import flow from manifest fetch through merge commit creation - Tar stream format and connection to deployments - Signature verification options - Key types and submodules container/store.rs: - Reference namespace constants and their purposes - Three-step import process (create, prepare, execute) - Layer types (commit, component, derived) and their handling - Merge commit metadata keys - Layer caching and deduplication strategy - Garbage collection behavior - Example usage lib.rs: - Add key modules section highlighting container, tar, sysroot, chunking This complements the recent installation documentation by explaining how container images are actually stored on disk in the ostree repository. Assisted-by: OpenCode (Claude Sonnet 4) Signed-off-by: Colin Walters <[email protected]>
1 parent b17ca33 commit 5c52b25

File tree

3 files changed

+283
-20
lines changed

3 files changed

+283
-20
lines changed

‎crates/ostree-ext/src/container/mod.rs‎

Lines changed: 150 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,164 @@
11
//! # APIs bridging OSTree and container images
22
//!
3-
//! This module contains APIs to bidirectionally map between a single OSTree commit and a container image wrapping it.
4-
//! Because container images are just layers of tarballs, this builds on the [`crate::tar`] module.
3+
//! This module provides the core infrastructure for bidirectionally mapping between
4+
//! OCI/Docker container images and OSTree repositories. It enables bootable container
5+
//! images to be fetched from registries, stored efficiently, and deployed as ostree
6+
//! commits.
57
//!
6-
//! To emphasize this, the current high level model is that this is a one-to-one mapping - an ostree commit
7-
//! can be exported (wrapped) into a container image, which will have exactly one layer. Upon import
8-
//! back into an ostree repository, all container metadata except for its digested checksum will be discarded.
8+
//! ## Overview
9+
//!
10+
//! Container images are fundamentally layers of tarballs. This module leverages the
11+
//! [`crate::tar`] module to import container layers as ostree content, and exports
12+
//! ostree commits back to container images. The key insight is that ostree's
13+
//! content-addressed object storage maps naturally to OCI layer deduplication.
14+
//!
15+
//! When a container image is imported ("pulled"), each layer becomes an ostree commit.
16+
//! These layer commits are then merged into a single "merge commit" that represents
17+
//! the complete filesystem state. This merge commit is what gets deployed as a
18+
//! bootable system.
19+
//!
20+
//! ## On-Disk Storage Structure
21+
//!
22+
//! Container images are stored in the ostree repository (typically `/sysroot/ostree/repo/`)
23+
//! using a structured reference (ref) namespace:
24+
//!
25+
//! ### Reference Namespace
26+
//!
27+
//! - **`ostree/container/blob/<escaped-digest>`**: Each OCI layer is stored as a
28+
//! separate ostree commit. The digest (e.g., `sha256:abc123...`) is escaped using
29+
//! [`crate::refescape`] to be valid as an ostree ref. For example:
30+
//! `ostree/container/blob/sha256_3A_abc123...`
31+
//!
32+
//! - **`ostree/container/image/<escaped-image-reference>`**: Points to the "merge
33+
//! commit" for a pulled image. The image reference (e.g., `docker://quay.io/org/image:tag`)
34+
//! is escaped similarly. This is the ref that deployments point to.
35+
//!
36+
//! - **`ostree/container/baseimage/<project>/<index>`**: Used to protect base images
37+
//! from garbage collection. Tooling that builds derived images locally should write
38+
//! refs under this prefix to prevent the base layers from being pruned.
39+
//!
40+
//! ### Layer Storage
41+
//!
42+
//! Each container layer is stored as an ostree commit with a special structure:
43+
//!
44+
//! - **OSTree "chunk" layers**: Layers that are part of the base ostree commit use
45+
//! the "object set" format - the filenames in the commit *are* the object checksums.
46+
//! This enables efficient reconstruction of the original ostree commit.
47+
//!
48+
//! - **Derived layers**: Non-ostree layers (e.g., from `RUN` commands in a Containerfile)
49+
//! are imported as regular filesystem trees and stored as standard ostree commits.
50+
//!
51+
//! ### The Merge Commit
52+
//!
53+
//! The merge commit (`ostree/container/image/...`) combines all layers into a single
54+
//! filesystem tree. It contains critical metadata in its commit metadata:
55+
//!
56+
//! - `ostree.manifest-digest`: The OCI manifest digest (e.g., `sha256:...`)
57+
//! - `ostree.manifest`: The complete OCI manifest as JSON
58+
//! - `ostree.container.image-config`: The OCI image configuration as JSON
59+
//!
60+
//! This metadata enables round-tripping: an imported image can be re-exported with
61+
//! its original manifest structure preserved.
62+
//!
63+
//! ## Import Flow
64+
//!
65+
//! The import process (implemented in [`store::ImageImporter`]) follows these steps:
66+
//!
67+
//! 1. **Manifest fetch**: Contact the registry via containers-image-proxy (skopeo)
68+
//! to retrieve the image manifest and configuration.
69+
//!
70+
//! 2. **Layout parsing**: Analyze the manifest to identify:
71+
//! - The base ostree layer (identified by the `ostree.final-diffid` label)
72+
//! - Component/chunk layers (split object sets)
73+
//! - Derived layers (non-ostree content)
74+
//!
75+
//! 3. **Layer caching check**: For each layer, check if an ostree ref already exists
76+
//! for that digest. Cached layers are skipped, enabling efficient incremental updates.
77+
//!
78+
//! 4. **Layer import**: For uncached layers:
79+
//! - Fetch the compressed tarball from the registry
80+
//! - Decompress and parse the tar stream
81+
//! - Import content into ostree (handling xattrs via `bare-split-xattrs` format)
82+
//! - Create an ostree commit and write the layer ref
83+
//!
84+
//! 5. **Merge commit creation**: Overlay all layers (processing OCI whiteout files)
85+
//! to create a unified filesystem tree. Apply SELinux labeling if needed.
86+
//! Store manifest/config metadata and write the image ref.
87+
//!
88+
//! 6. **Garbage collection**: Prune layer refs that are no longer referenced by any
89+
//! image or deployment.
90+
//!
91+
//! ## Tar Stream Format
92+
//!
93+
//! The tar format used for ostree layers is documented in [`crate::tar`]. Key points:
94+
//!
95+
//! - Uses `bare-split-xattrs` repository mode to handle extended attributes
96+
//! - XAttrs are stored in separate `.file-xattrs` objects, avoiding tar xattr complexity
97+
//! - `/etc` in container images maps to `/usr/etc` in ostree (the "3-way merge" location)
98+
//! - Hardlinks are used for deduplication within layers
99+
//!
100+
//! ## Connection to Deployments
101+
//!
102+
//! When bootc deploys an image, it creates an ostree deployment whose "origin" file
103+
//! references the container image. The origin contains:
104+
//!
105+
//! - The [`OstreeImageReference`] specifying the image and signature verification method
106+
//! - The merge commit checksum
107+
//!
108+
//! On subsequent boots, bootc can compare the deployed commit against the registry
109+
//! manifest to detect available updates.
9110
//!
10111
//! ## Signatures
11112
//!
12-
//! OSTree supports GPG and ed25519 signatures natively, and it's expected by default that
13-
//! when booting from a fetched container image, one verifies ostree-level signatures.
14-
//! For ostree, a signing configuration is specified via an ostree remote. In order to
15-
//! pair this configuration together, this library defines a "URL-like" string schema:
113+
//! OSTree supports GPG and ed25519 signatures natively. When fetching container images,
114+
//! signature verification can be configured via [`SignatureSource`]:
115+
//!
116+
//! - `OstreeRemote(name)`: Verify using the named ostree remote's keyring
117+
//! - `ContainerPolicy`: Defer to containers-policy.json (requires explicit allow)
118+
//! - `ContainerPolicyAllowInsecure`: Use containers-policy.json defaults (not recommended)
119+
//!
120+
//! This library defines a URL-like schema to combine signature verification with
121+
//! image references:
122+
//!
123+
//! - `ostree-remote-registry:<remotename>:<containerimage>` - Verify via ostree remote
124+
//! - `ostree-image-signed:<transport>:<image>` - Use container policy
125+
//! - `ostree-unverified-registry:<image>` - No verification (not recommended)
126+
//!
127+
//! Example: `ostree-remote-registry:fedora:quay.io/fedora/fedora-bootc:latest`
128+
//!
129+
//! See [`OstreeImageReference`] for parsing and generating these strings.
130+
//!
131+
//! ## Layering and Derived Images
132+
//!
133+
//! Container image layering is fully supported. A typical bootable image structure:
134+
//!
135+
//! 1. **Base ostree layer**: Contains the core OS as an ostree commit
136+
//! 2. **Chunk layers**: Split objects for efficient updates (optional)
137+
//! 3. **Derived layers**: Additional content from Containerfile `RUN` commands
138+
//!
139+
//! The `ostree.final-diffid` label in the image configuration marks where the
140+
//! ostree content ends and derived content begins. This enables:
16141
//!
17-
//! `ostree-remote-registry:<remotename>:<containerimage>`
142+
//! - Efficient layer sharing between images with the same base
143+
//! - Proper SELinux labeling of derived content using the base policy
144+
//! - Round-trip export preserving the layer structure
18145
//!
19-
//! A concrete instantiation might be e.g.: `ostree-remote-registry:fedora:quay.io/coreos/fedora-coreos:stable`
146+
//! ## Key Types
20147
//!
21-
//! To parse and generate these strings, see [`OstreeImageReference`].
148+
//! - [`Transport`]: OCI/Docker transport (registry, oci-dir, containers-storage, etc.)
149+
//! - [`ImageReference`]: Container image reference with transport
150+
//! - [`OstreeImageReference`]: Image reference plus signature verification method
151+
//! - [`SignatureSource`]: How to verify image signatures
152+
//! - [`store::ImageImporter`]: Main import orchestrator
153+
//! - [`store::PreparedImport`]: Analysis of layers to fetch
154+
//! - [`store::LayeredImageState`]: State of a pulled image
155+
//! - [`ManifestDiff`]: Comparison between two image manifests
22156
//!
23-
//! ## Layering
157+
//! ## Submodules
24158
//!
25-
//! A key feature of container images is support for layering. At the moment, support
26-
//! for this is [planned but not implemented](https://github.com/ostreedev/ostree-rs-ext/issues/12).
159+
//! - [`store`]: Core storage and import logic
160+
//! - [`deploy`]: Integration with ostree deployments
161+
//! - [`skopeo`]: Skopeo subprocess management for registry operations
27162
28163
use anyhow::anyhow;
29164
use cap_std_ext::cap_std;

‎crates/ostree-ext/src/container/store.rs‎

Lines changed: 124 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,129 @@
1-
//! APIs for storing (layered) container images as OSTree commits
1+
//! # Storing layered container images as OSTree commits
22
//!
3-
//! # Extension of encapsulation support
3+
//! This module implements the core storage and import logic for container images in
4+
//! ostree. It handles fetching images from registries, caching layers as ostree commits,
5+
//! and creating merge commits that represent the complete image filesystem.
46
//!
5-
//! This code supports ingesting arbitrary layered container images from an ostree-exported
6-
//! base. See [`encapsulate`][`super::encapsulate()`] for more information on encapsulation of images.
7+
//! ## Overview
8+
//!
9+
//! The primary entry point is [`ImageImporter`], which orchestrates the import of a
10+
//! container image. The import process efficiently handles incremental updates by
11+
//! caching each layer as a separate ostree commit, only fetching layers that aren't
12+
//! already present.
13+
//!
14+
//! ## Reference Namespace Constants
15+
//!
16+
//! Layers and images are stored using these ref prefixes (defined as constants in this module):
17+
//!
18+
//! - `ostree/container/blob`: Individual OCI layers stored as commits
19+
//! - `ostree/container/image`: Merge commits for complete images
20+
//! - [`BASE_IMAGE_PREFIX`] (`ostree/container/baseimage`): Protected base images (public)
21+
//!
22+
//! Layer refs use escaped digests (e.g., `sha256:abc...` becomes `sha256_3A_abc...`)
23+
//! via [`crate::refescape`] to conform to ostree ref naming requirements.
24+
//!
25+
//! ## Import Process
26+
//!
27+
//! A typical import flow:
28+
//!
29+
//! 1. **Create importer**: [`ImageImporter::new`] initializes the proxy connection
30+
//! to the container registry (via containers-image-proxy/skopeo).
31+
//!
32+
//! 2. **Prepare import**: [`ImageImporter::prepare`] fetches the manifest and
33+
//! analyzes which layers need to be downloaded:
34+
//! - Returns [`PrepareResult::AlreadyPresent`] if the image is unchanged
35+
//! - Returns [`PrepareResult::Ready`] with a [`PreparedImport`] containing
36+
//! the download plan
37+
//!
38+
//! 3. **Execute import**: [`ImageImporter::import`] downloads missing layers and
39+
//! creates the merge commit:
40+
//! - Each layer is fetched, decompressed, and imported as an ostree commit
41+
//! - The merge commit overlays all layers, processing whiteouts
42+
//! - Image metadata (manifest, config) is stored in commit metadata
43+
//!
44+
//! ## Layer Types
45+
//!
46+
//! The manifest layout is parsed to identify different layer types:
47+
//!
48+
//! - **Commit layer**: The base ostree commit layer (identified by `ostree.final-diffid`)
49+
//! - **Component layers**: Additional ostree "chunk" layers containing split objects
50+
//! - **Derived layers**: Non-ostree layers from Containerfile `RUN` commands
51+
//!
52+
//! Each layer type is handled differently during import:
53+
//!
54+
//! - Ostree layers use object-set import mode for efficient reconstruction
55+
//! - Derived layers are imported as regular filesystem trees with SELinux labeling
56+
//!
57+
//! ## Merge Commit Metadata
58+
//!
59+
//! The merge commit stores essential metadata for image management:
60+
//!
61+
//! - `ostree.manifest-digest`: The canonical manifest digest (e.g., `sha256:...`)
62+
//! - `ostree.manifest`: Complete OCI manifest as canonical JSON
63+
//! - `ostree.container.image-config`: OCI image configuration as canonical JSON
64+
//!
65+
//! This metadata enables:
66+
//! - Detecting when updates are available
67+
//! - Re-exporting images with preserved structure
68+
//! - Querying image state via [`query_image`] and [`query_image_commit`]
69+
//!
70+
//! ## Layer Caching and Deduplication
71+
//!
72+
//! Layers are cached by their content digest, enabling:
73+
//!
74+
//! - **Incremental updates**: Only changed layers are downloaded
75+
//! - **Cross-image sharing**: Images sharing layers reuse cached commits
76+
//! - **Efficient storage**: Ostree's content-addressed storage deduplicates files
77+
//!
78+
//! The `query_layer` function checks if a layer is already cached by looking up
79+
//! its ref. During import, cached layers are skipped entirely.
80+
//!
81+
//! ## Garbage Collection
82+
//!
83+
//! Unreferenced layers are automatically pruned after imports via [`gc_image_layers`]:
84+
//!
85+
//! 1. Collect all layer digests referenced by stored images and deployments
86+
//! 2. List all layer refs under `ostree/container/blob/`
87+
//! 3. Remove refs for layers not in the referenced set
88+
//!
89+
//! Note: This only removes refs; actual object pruning requires a separate
90+
//! call to `ostree::Repo::prune`.
91+
//!
92+
//! ## Key Types
93+
//!
94+
//! - [`ImageImporter`]: Main import orchestrator with progress tracking
95+
//! - [`PrepareResult`]: Result of preparing an import (already present vs. ready)
96+
//! - [`PreparedImport`]: Detailed import plan with layer analysis
97+
//! - [`ManifestLayerState`]: Per-layer state (descriptor, ref, cached commit)
98+
//! - [`LayeredImageState`]: Complete state of a pulled image
99+
//! - [`CachedImageUpdate`]: Cached metadata for pending updates
100+
//! - [`ImportProgress`]: Progress events for layer fetches
101+
//! - [`LayerProgress`]: Byte-level progress for a single layer
102+
//!
103+
//! ## Example Usage
104+
//!
105+
//! ```ignore
106+
//! use ostree_ext::container::{OstreeImageReference, store::ImageImporter};
107+
//!
108+
//! let imgref: OstreeImageReference = "ostree-unverified-registry:quay.io/fedora/fedora-bootc:latest".parse()?;
109+
//! let mut importer = ImageImporter::new(&repo, &imgref, Default::default()).await?;
110+
//!
111+
//! match importer.prepare().await? {
112+
//! PrepareResult::AlreadyPresent(state) => {
113+
//! println!("Image already at {}", state.manifest_digest);
114+
//! }
115+
//! PrepareResult::Ready(prep) => {
116+
//! println!("Fetching {} layers", prep.layers_to_fetch().count());
117+
//! let state = importer.import(prep).await?;
118+
//! println!("Imported {}", state.merge_commit);
119+
//! }
120+
//! }
121+
//! ```
122+
//!
123+
//! ## See Also
124+
//!
125+
//! - [`super::encapsulate`]: Export ostree commits to container images
126+
//! - [`crate::tar`]: Tar stream format for layer content
7127
8128
use super::*;
9129
use crate::chunking::{self, Chunk};

‎crates/ostree-ext/src/lib.rs‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,15 @@
22
//!
33
//! This crate builds on top of the core ostree C library
44
//! and the Rust bindings to it, adding new functionality
5-
//! written in Rust.
5+
//! written in Rust.
6+
//!
7+
//! ## Key Modules
8+
//!
9+
//! - [`container`]: Bidirectional mapping between OCI container images and ostree commits.
10+
//! This is the core of bootc's ability to deploy container images as bootable systems.
11+
//! - [`tar`]: Lossless export and import of ostree commits as tar archives.
12+
//! - [`sysroot`]: Extensions for managing ostree deployments.
13+
//! - [`chunking`]: Splitting ostree commits into layers for efficient container updates.
614
715
// See https://doc.rust-lang.org/rustc/lints/listing/allowed-by-default.html
816
#![deny(missing_docs)]

0 commit comments

Comments
 (0)