Design layered ASIF images for CI image variants

[pepicrft]

Context

Tart's current model is intentionally simple:

• A local VM is a directory with config.json, nvram.bin, and disk.img.
• A remote VM is an OCI reference, for example ghcr.io/org/image:tag.
• tart clone creates a local VM and relies on APFS copy-on-write when available.
• Tart's current OCI disk layers are transfer chunks, not semantic VM image layers like "base macOS", "Xcode", or "iOS simulator runtime".

That works well for local clones, but CI image distribution has a different pressure point. Images such as "macOS + Xcode" and "macOS + Xcode + simulator runtime" share most of their disk content, but today we still tend to publish and consume them as full image variants.

WWDC26 introduced DiskImageKit stacked ASIF images and VZDiskImageStorageDeviceAttachment(diskImage:). That gives Tart a public Apple API for block-level image stacks:

base ASIF image
+ read-only child ASIF image
+ writable overlay ASIF image

This is close to the image/template layering model used by systems such as Anka, VMware linked clones, Parallels linked clones, Hyper-V differencing disks, QEMU backing chains, and Docker/OCI image layers. The common pattern is that users think in images, templates, or tags, while layers are the storage and distribution mechanism.

Proposal

Keep Tart's user-facing concepts as VMs and images. Do not introduce tart layer as a top-level concept initially.

Use layers internally as immutable ASIF artifacts referenced by image manifests and cached by digest.

A CI image variant would look like this internally:

A: macOS 27 + Xcode 27
B: delta from A after installing iOS 27 simulator runtime
C: delta from A after installing watchOS 27 simulator runtime
D: delta from B after installing watchOS 27 simulator runtime

Registry tags would point at ordered chains:

ghcr.io/tuist/macos-xcode:27              -> A
ghcr.io/tuist/macos-xcode:27-ios-27       -> A + B
ghcr.io/tuist/macos-xcode:27-watch-27     -> A + C
ghcr.io/tuist/macos-xcode:27-ios-watch-27 -> A + B + D

Consumers pull image tags, not individual layers:

tart pull ghcr.io/tuist/macos-xcode:27-ios-27
tart run --ephemeral ghcr.io/tuist/macos-xcode:27-ios-27

Tart reads the manifest, downloads only the missing layer blobs, validates the parent chain, and assembles the runtime disk as:

A read-only
+ B read-only
+ temporary writable job overlay

Producer flow

The build side needs to publish layered image tags deliberately.

Example:

tart push --layered base-vm ghcr.io/tuist/macos-xcode:27

tart clone ghcr.io/tuist/macos-xcode:27 with-ios-27
# install iOS simulator runtime in with-ios-27
tart push --layered with-ios-27 ghcr.io/tuist/macos-xcode:27-ios-27

Expected semantics:

• If the VM has no layered parent, push --layered publishes its disk as a base ASIF layer.
• If the VM was created from a layered image, push --layered publishes only its writable overlay as a child layer.
• The child layer records the exact parent digest it was built from.
• Published layers are immutable.

Consumer flow

Consumers should not have to know which layers exist.

tart pull ghcr.io/tuist/macos-xcode:27-ios-27

If the machine already has the base image cached, Tart should only pull the simulator delta.

tart run --ephemeral ghcr.io/tuist/macos-xcode:27-ios-27

Tart should create a temporary writable overlay, attach the full stack with DiskImageKit, and delete the writable overlay after shutdown.

A later, optional UX could resolve named flavors:

tart run --ephemeral ghcr.io/tuist/macos-xcode:27 --with ios-27

But this should resolve to a prebuilt compatible image chain. Tart should not blindly combine arbitrary ASIF layers.

Safety rules

• Layered images require ASIF and macOS 27+.
• Parent layers are immutable and opened read-only.
• Every child layer records and validates its exact parent digest.
• Tart refuses to assemble a chain if any parent digest is missing or mismatched.
• tart set --disk-size on layered VMs should fail or require flattening first.
• tart set --disk on layered VMs should fail or require flattening first.
• tart push keeps the current flat behavior unless --layered is passed.
• Layer stack depth should be bounded or warnings should be emitted because deep stacks can affect performance and recovery.
• Cached layers need reference-aware garbage collection so a base layer is not deleted while images or VMs still depend on it.

Storage model

Potential local cache layout:

~/.tart/cache/disk-layers/sha256/<digest>.asif
~/.tart/cache/disk-layers/sha256/<digest>.json

A local layered VM could remain compatible with Tart's existing VM directory shape by using disk.img for the writable overlay and adding stack metadata:

~/.tart/vms/job-vm/
  config.json
  nvram.bin
  disk.img          # writable overlay
  disk-stack.json   # ordered references to cached immutable layers

At run time, Tart would build the DiskImageKit.DiskImage stack from disk-stack.json and pass it to VZDiskImageStorageDeviceAttachment(diskImage:).

OCI shape

We should keep the current flat Tart OCI format for compatibility and add a new layered ASIF format alongside it.

The image manifest should include:

• ordered disk layer descriptors, bottom to top
• layer digest
• parent digest, nil only for the base layer
• ASIF metadata needed to validate/open the layer
• config and NVRAM blobs
• annotations identifying the format as Tart layered ASIF

This keeps normal OCI pull behavior: if a layer digest already exists locally, Tart skips downloading it.

Operations to design

Initial commands worth considering:

tart run --ephemeral <image>
tart push --layered <vm> <image>
tart flatten <vm>
tart pull <image>

flatten is important because linked/layered systems all need an escape hatch that turns a dependent VM into a standalone disk.

Suggested implementation phases

1. Add metadata and runtime support for assembling a local DiskImageKit stack.
2. Add tart run --ephemeral for layered images with a temporary writable overlay.
3. Add layered OCI pull support that caches immutable ASIF layers by digest.
4. Add tart push --layered for producing base and child image tags.
5. Add tart flatten and reference-aware cache pruning.

Open questions

• Should layered images store NVRAM per tag, per VM instance, or both?
• Should tart clone <layered-image> <vm> create a persistent writable overlay VM, or should persistent clones remain flat by default?
• How should Tart represent multi-architecture layered images in OCI?
• Should simulator flavors be explicit tags only, or should Tart later support a --with <flavor> resolver?
• What is the maximum safe stack depth before Tart should suggest flattening?
• Can the first version avoid changing the existing flat OCI media types entirely by introducing separate media types for layered ASIF images?