A pure-Swift container build system

[katiewasnothere]

Many of the container maintainers and I have been working on creating a native container builder to eventually replace our https://github.com/apple/container-builder-shim use. I've opened a PR here #399 with our initial design and partial implementations. There are some docs included in that PR that describe our approach and some of our goals.

We'd very much like to make this a community effort and we're open to all feedback on the design and initial work!

@wlan0

[wlan0]

Thanks for kicking this off, @katiewasnothere !

For everyone, the design docs bundled in PR #399 outline our goals, the proposed build-graph model, and the initial implementation. We’d love your eyes on those details. Especially around layer caching, multi-arch strategy, and dependency resolution.

If there are capabilities you’d like us to consider early (e.g., registry push/pull hooks, progress UI, SBOM generation), please drop them here. The sooner we know your priorities, the better we can shape the roadmap.

Looking forward to your feedback and to building this together to make container builds in Swift a first-class experience! 🚀

[wlan0]

Hi everyone, we've been steadily progressing with the pure-swift builder implementation. The IR and build graph were introduced in the first commit. Then, we achieved a good amount of coverage of Dockerfile instructions in the parser. The differ/snapshotter impl in progress. Here is a detailed design/implementation proposal for implementing Filesystem Executor. We are following a unique approach for snapshotting image layers into a single EXT4 block device. This has been made possible, thanks to our earlier work on the pure-swift EXT4 formatter.

TL;DR: The filesystem executor talks directly to pure‑Swift EXT4 stack from github.com/apple/containerization. Concretely, mapping FilesystemOperationExecutor to ContainerizationEXT4.EXT4.Formatter for in‑place population of the rootfs block device (files, dirs, symlinks, hardlinks, xattrs, timestamps) and using EXT4.Formatter.unpack(...) to stream‑extract tars for ADD is the proposal. This way, there is no need to mount the block device on macOS or Linux, instead writing into the block file that becomes the VM’s root disk is more efficient and straightforward. The Containerization API explicitly exposes modules for creating and populating EXT4 filesystems without mounting them (EXT4.Formatter.create(...), EXT4.Formatter.unpack(...), EXT4.EXT4Reader), so the executor can remain 100% Swift with no FUSE or /sbin/mount in the loop.

High level design

• Consumes FilesystemOperation IR (COPY/ADD/remove/mkdir/symlink/hardlink)
• Writes directly into the container’s EXT4 rootfs block (the layer/snapshot we’re preparing)
• Preserves Docker semantics (ADD auto‑extracts archives, COPY doesn’t)
• Enforces sandbox/path security
• Emits a diff to Snapshotter/Differ for layer creation and caching

Containerization EXT4’s design exposes the container filesystem as a block device and ships a Swift EXT4 library to format and populate that device, which is exactly what our executor needs.

Mapping Containerization’s EXT4 APIs to FS Executor

Containerization EXT4 module ContainerizationEXT4 with EXT4 types are detailed below:

• EXT4 – “read the superblock of an existing EXT4 block device and format a new block device” (we’ll use this for initial empty snapshots).

• EXT4.EXT4Reader – opens a device, parses superblock, group descriptors, inodes (useful for validations and diff sanity checks).

• EXT4.Formatter – high‑level populator with:

  • create(path:link:mode:ts:buf:uid:gid:xattrs:recursion:) — write regular files (via buf), make directories, symlinks, hardlinks, set perms/ownership/timestamps/xattrs. (This is our main “COPY into filesystem” primitive.)
  • unpack(source:format:compression:progress:) — stream‑extract an archive into the EXT4 image (this is ADD with tar/zip/… handling).

Executor architecture

Dockerfile -> Parser -> FilesystemOperation IR
                 |
                 v
        FilesystemOperationExecutor
                 |
   +-------------+--------------+
   |             |              |
  COPY          ADD           Advanced ops
   |             |              |
   v             v              v
Ext4Write (Formatter.create)  mkdir/symlink/hardlink/remove -> Formatter.create/remove
ADD url/tar -> Formatter.unpack

• We never mount the rootfs on macOS; we treat it as a file descriptor for the EXT4 block file and operate via EXT4 APIs. The VM later boots with that disk

API

The following protocol is defined in such a way that it can be mocked for tests:

// A thin wrapper over ContainerizationEXT4
public protocol EXT4WritableFS: Sendable {
    func create(
        path: String,
        link: String?,           // symlink or hardlink target (see semantics below)
        mode: UInt16,            // POSIX mode (0o755, etc.)
        ts: (atime: UInt64, mtime: UInt64, ctime: UInt64)?,
        buf: UnsafeRawBufferPointer?, // nil for non-regular files
        uid: UInt32,
        gid: UInt32,
        xattrs: [String: Data]?,
        recursion: Bool
    ) throws

    func unpack(
        sourceFD: Int32,         // tar/zip stream (seekable not required)
        format: ArchiveFormat,   // .tar, .zip, ...
        compression: Compression // .none, .gzip, .bzip2, .xz
    ) throws
}

public enum ArchiveFormat: Sendable { case tar, zip }       // map 1:1 to EXT4.Formatter enums
public enum Compression: Sendable { case none, gzip, bzip2, xz }

These signatures line up with EXT4.Formatter.create(...) and unpack(source:format:compression:...) from Containerization.

Adapter

The adapter bridges EXT4Formatter and EXT4WritableFS:

import ContainerizationEXT4
import Foundation

public final class Ext4FormatterAdapter: Ext4WritableFS {
    private let formatter: EXT4.Formatter

    // obtain `formatter` from snapshotter open/prepare step.
    public init(formatter: EXT4.Formatter) { self.formatter = formatter }

    public func create(
        path: String,
        link: String?,
        mode: UInt16,
        ts: (atime: UInt64, mtime: UInt64, ctime: UInt64)?,
        buf: UnsafeRawBufferPointer?,
        uid: UInt32,
        gid: UInt32,
        xattrs: [String : Data]?,
        recursion: Bool
    ) throws {
        // Direct pass-through to ContainerizationEXT4
        try formatter.create(
            path: path,
            link: link,
            mode: mode,
            ts: ts,
            buf: buf,
            uid: uid,
            gid: gid,
            xattrs: xattrs,
            recursion: recursion
        )
    }

    public func unpack(sourceFD: Int32, format: ArchiveFormat, compression: Compression) throws {
        // Map our enums to the ContainerizationEXT4 enums.
        let fmt: EXT4.Formatter.ArchiveFormat = (format == .tar) ? .tar : .zip
        let cmp: EXT4.Formatter.Compression
        switch compression {
        case .none:  cmp = .none
        case .gzip:  cmp = .gzip
        case .bzip2: cmp = .bzip2
        case .xz:    cmp = .xz
        }
        try formatter.unpack(source: .fileDescriptor(sourceFD), format: fmt, compression: cmp, progress: nil)
    }
}

Wiring into FilesystemOperationExecutor

In FilesystemOperationExecutor, replace POSIX host FS writes with calls into EXT4WritableFS:

• COPY (context → dest)
  For each source path, stream file contents and call ext4.create(...) with:

  • path: normalized destination relative to container root (/ in the image)
  • mode: from metadata or source
  • ts: timestamps (if enabled)
  • buf: raw bytes for regular files; nil for dirs/symlinks/hardlinks
  • uid/gid: from --chown or default
  • xattrs: preserved when preserveXattrs
  • recursion: true when copying a directory tree

• ADD (context URL/archives → dest)

  • If source is a tar/zip and ADD, open an InputStream/fd and ext4.unpack(...) with the right format/compression.
  • If source is a regular file, treat like COPY (no auto‑extract under COPY).

• mkdir/symlink/hardlink/remove

  • mkdir: create(path:..., buf: nil, mode: 0o755, recursion: true)
  • symlink: create(path:..., link: "<target>", buf: nil, mode: 0o777)
  • hardlink: create(path:..., link: "<existingPath>", buf: nil, mode: 0o777)
  • remove: call create(...) with a deletion sentinel is not correct. Instead do a real delete via EXT4 writer (Containerization’s EXT4 API exposes full FS manipulation). Fall back to snapshot differ to represent deletes.

The availability of hardlinks/symlinks and their error surface (e.g. “cannot create hardlinks to dir target”) show up in EXT4.Formatter.Error.*—propagate those as structured errors.

1) Path sandboxing (prevents .. & symlink escapes)

import Foundation

/// Returns a normalized absolute path under `/` for the image.
/// Throws if the resolved path escapes the container root.
func normalizedContainerPath(_ dest: String) throws -> String {
    // Disallow absolute host paths; our root is always "/".
    var p = dest
    if !p.hasPrefix("/") { p = "/" + p }
    // RFC 3986-ish normalization: collapse //, /./ and resolve ../
    let parts = p.split(separator: "/").reduce(into: [String]()) { acc, el in
        switch el {
        case "", ".": break
        case "..": if !acc.isEmpty { _ = acc.popLast() }
        default: acc.append(String(el))
        }
    }
    let normalized = "/" + parts.joined(separator: "/")
    guard normalized.hasPrefix("/") else { throw ExecutorError.pathTraversal(dest) }
    return normalized
}

2) COPY file into EXT4 with metadata

import Foundation

struct CopySpec {
    let srcURL: URL
    let destPath: String
    let uid: UInt32
    let gid: UInt32
    let mode: UInt16
    let xattrs: [String: Data]?
    let timestamps: (UInt64, UInt64, UInt64)? // (atime, mtime, ctime) seconds since epoch
}

func copyFile(_ spec: CopySpec, into fs: EXT4WritableFS) throws {
    let dest = try normalizedContainerPath(spec.destPath)

    let data = try Data(contentsOf: spec.srcURL, options: .mappedIfSafe)
    try data.withUnsafeBytes { raw in
        try fs.create(
            path: dest,
            link: nil,
            mode: spec.mode,
            ts: spec.timestamps.map { (at, mt, ct) in (atime: at, mtime: mt, ctime: ct) },
            buf: raw,
            uid: spec.uid,
            gid: spec.gid,
            xattrs: spec.xattrs,
            recursion: false
        )
    }
}

3) ADD tar.gz via streaming unpack

import System   // for FileDescriptor

func addTarGz(_ archiveURL: URL, dest: String, into fs: EXT4WritableFS) throws {
    // Destination is the working directory inside the image; EXT4.Formatter.unpack
    // maintains relative paths from the archive entries.
    _ = try normalizedContainerPath(dest) // validation only

    let fd = try FileDescriptor.open(archiveURL.path, .readOnly)
    defer { try? fd.close() }

    try fs.unpack(sourceFD: fd.rawValue, format: .tar, compression: .gzip)
}

The unpack(source:format:compression:...) call is the exact API for ADD auto‑extraction.

Snapshotter and “formatting” the block

• Initial empty snapshot: use Containerization’s EXT4 to format the block device that backs MutableSnapshot (choose block size, label, features). The module’s overview explicitly says EXT4 formats block devices.
• Open for mutation: hand an EXT4.Formatter (writable handle) to the executor for the duration of the step. After writes, commit the MutableSnapshot and compute a diff via Differ.

If read‑side sanity checks are needed (e.g., ensure we didn’t escape root), open EXT4.EXT4Reader on the same device and assert inode paths exist

Mapping from IR to EXT4 (summary)

IR action	EXT4.Formatter call	Notes
copy file	create(path:..., buf: <bytes>)	Set mode/uid/gid/xattrs/ts
copy dir	create(path:..., buf: nil, recursion: true)	Pre‑create dir before children
add tar(.gz/.bz2/.xz)	unpack(source:fd, format:.tar, compression:...)	Docker ADD behavior
add regular file	same as copy	No auto‑extract
mkdir	create(path:..., buf: nil, recursion: true)	Respect Permissions
symlink	create(path:..., link:"target", buf:nil)	mode isn’t meaningfully applied
hardlink	create(path:..., link:"/existing", buf:nil)	Error if link target is a dir (see Error enum)
remove	(EXT4 delete)	Use EXT4 delete primitive; not shown here

Security model

1. Path traversal & canonicalization — Always run normalizedContainerPath.
2. Symlink escape — Resolve any symlinks in the source before copying; treat link targets as opaque bytes in the destination (don’t resolve to host).
3. Archive safety — For unpack, deny absolute and .. entries; rely on EXT4.Formatter.unpack’s validation, and still double‑normalize per entry to defense‑in‑depth.
4. Permissions — Validate octal modes, deny setuid/setgid unless explicitly allowed by policy.
5. Xattrs — Bound total xattr bytes per file and per layer; apply only when preserveXattrs is set.

Concurrency & performance

• Use a TaskGroup with a global cap (10) to parallelize independent file materializations, but serialize calls that must be ordered (e.g., creating parent dirs before children).
• Stream large inputs (ADD archives or >64MiB files) straight into create/unpack (no whole‑blob buffering).
• Cache keys must include: base snapshot ID, source digests (content hash), dest paths, metadata flags, and follow‑symlink policy.

Errors & diagnostics

Translate EXT4 errors to structured error model:

• EXT4.Formatter.Error.alreadyExists → conflict under fail strategy
• EXT4.Formatter.Error.cannotCreateHardlinksToDirTarget → invalid hardlink target
  Propagate the IR’s OperationMetadata (Dockerfile location) onto every thrown error for precise messages.

Snapshotter & Differ glue

The snapshotter already exposes prepare/commit and a Differ. Just ensure the writable snapshot exposes an fd (or path) to the backing block file so we can create an EXT4.Formatter on it and hand an EXT4WritableFS to the executor. (Containerization’s repos and docs describe the block‑device model and module split into ContainerizationOCI, ContainerizationEXT4, ContainerizationArchive.)

Docker compatibility edge‑cases

• ADD of *.tar, *.tar.gz, *.tgz, *.tar.xz, *.zip → auto‑extract with path filtering
• COPY of archives → no auto‑extract
• --chown UID/GID by name vs numeric (resolve with /etc/passwd from build stage if present)
• --chmod octal parse & recursive application
• Symlink loops, absolute symlinks (/usr/bin/python) remain links, not resolved to host
• Hardlinks that target directories → error (propagate Formatter.Error)