fix: enforce closed-world bundle verification (NVIDIA#1758)

Signed-off-by: Mark Chmarny <mark@chmarny.com>

Author: mchmarny

demos/bundle-attestation-demo-slides.html

@@ -190,8 +190,8 @@ <h2>Generated artifacts have no upstream tag</h2>
       </tbody>
     </table>
     <div class="grid2">
-      <div class="mini"><h4>Without attestation</h4><p>Bundle is just files. <code class="inl">checksums.txt</code> catches accidental corruption, nothing else.</p></div>
-      <div class="mini"><h4>With <code class="inl">--attest</code></h4><p>The generated payload files listed in <code class="inl">checksums.txt</code>, including <code class="inl">recipe.yaml</code>, are cryptographically bound to the creator <em>and</em> the binary that produced it. Recursive provenance.</p></div>
+      <div class="mini"><h4>Without attestation</h4><p><code class="inl">checksums.txt</code> detects accidental payload or tree changes, but provides no signed creator identity or provenance.</p></div>
+      <div class="mini"><h4>With <code class="inl">--attest</code></h4><p>The complete closed-world payload inventory, including <code class="inl">recipe.yaml</code>, is cryptographically bound to the creator <em>and</em> the binary that produced it. Recursive provenance.</p></div>
     </div>
   </div>
 </section>
@@ -202,7 +202,7 @@ <h2>Generated artifacts have no upstream tag</h2>
     <div class="label">02 · Produce</div>
     <h2><code class="inl">aicr bundle --attest</code></h2>
     <figure class="fig">
-      <svg class="diagram" viewBox="0 0 860 380" role="img" aria-label="Recipe flows through aicr bundle --attest, producing a bundle directory with attestation/ subdirectory containing two signed Sigstore bundles">
+      <svg class="diagram" viewBox="0 0 860 380" role="img" aria-label="Recipe flows through aicr bundle --attest, producing a closed-world bundle inventory with checksums.txt and the two permitted attestation files">
         <defs>
           <marker id="bD1" markerWidth="9" markerHeight="9" refX="7" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 Z" fill="#46525f"/></marker>
           <marker id="bD1g" markerWidth="9" markerHeight="9" refX="7" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 Z" fill="#76b900"/></marker>
@@ -233,7 +233,8 @@ <h2><code class="inl">aicr bundle --attest</code></h2>
           <text x="46" y="214">recipe.yaml &#183; deploy.sh &#183; README.md</text>
           <text x="46" y="234">&lt;component&gt;/values.yaml &#183; ...</text>
           <text x="46" y="266" fill="#9aa7b4">checksums.txt</text>
-          <text x="46" y="282" fill="#6b7785" font-size="11">(SHA256 of every generated payload file)</text>
+          <text x="46" y="282" fill="#6b7785" font-size="11">closed-world SHA256 inventory; covers recipe.yaml</text>
+          <text x="46" y="300" fill="#6b7785" font-size="11">additional filesystem entries are rejected</text>
         </g>
         <rect class="node-g" x="432" y="170" width="398" height="186" rx="9"/>
         <text class="ttl-g nlabel" x="448" y="194">attestation/</text>
@@ -300,7 +301,9 @@ <h2>Bundle &harr; binary &harr; NVIDIA CI</h2>
       <span class="t info">Why both halves matter</span>
       The bundle signature alone proves a creator made a bundle. The chain to the binary
       attestation proves the bundle was built by an attested NVIDIA-CI-released AICR CLI —
-      not by a fork, an alpha build, or a tampered binary.
+      not by a fork, an alpha build, or a tampered binary. AICR generates manifest entries
+      sorted, but validates valid entries in any order; reordering a signed manifest changes its
+      signed bytes and invalidates this chain.
     </div>
   </div>
 </section>
@@ -322,8 +325,9 @@ <h2><code class="inl">aicr verify</code> — clean pass</h2>
 
 <span class="g">Bundle verification: PASSED</span></pre>
     </div>
-    <p>Five gates ran: file integrity, bundle signature, bundle predicate, binary attestation
-      chain, identity pin. Any failure short-circuits to a lower trust level.</p>
+    <p>Five gates ran: closed-world inventory, bundle signature, bundle predicate, binary
+      attestation chain, identity pin. Additional files or directories, symlinks, other
+      non-regular objects, and failed attestations report <code class="inl">unknown</code>.</p>
   </div>
 </section>
 
@@ -336,16 +340,17 @@ <h2>Four trust levels</h2>
       <thead><tr><th>Level</th><th>Name</th><th>Criteria</th></tr></thead>
       <tbody>
         <tr><td class="ok">4</td><td><code>verified</code></td><td>Checksums &nbsp;+ &nbsp;bundle attestation &nbsp;+ &nbsp;binary attestation pinned to NVIDIA CI</td></tr>
-        <tr><td class="ok">3</td><td><code>attested</code></td><td>Bundle attestation verified; binary attestation missing or external data used (a <em>failed</em> binary attestation reports attested but exits nonzero)</td></tr>
-        <tr><td class="warn">2</td><td><code>unverified</code></td><td>Checksums valid, no <code class="inl">--attest</code> was passed</td></tr>
-        <tr><td class="bad">1</td><td><code>unknown</code></td><td>Missing/invalid <code class="inl">checksums.txt</code>, or bundle attestation fails verification</td></tr>
+        <tr><td class="ok">3</td><td><code>attested</code></td><td>Bundle attestation verified; binary attestation missing, or external data used</td></tr>
+        <tr><td class="warn">2</td><td><code>unverified</code></td><td>Closed-world checksum inventory valid, no <code class="inl">--attest</code> was passed</td></tr>
+        <tr><td class="bad">1</td><td><code>unknown</code></td><td>Missing, invalid, or incomplete manifest; unexpected filesystem entry; or failed attestation</td></tr>
       </tbody>
     </table>
     <div class="note">
       <span class="t info">Why levels, not pass/fail</span>
       Different bundles live under different threats. CI-built production bundles must be
       <code class="inl">verified</code>; an on-call engineer&rsquo;s emergency hotfix bundle might only
-      reach <code class="inl">attested</code>. Operators pick the floor per environment.
+      reach <code class="inl">attested</code>. Legacy bundles with incomplete manifests remain
+      <code class="inl">unknown</code> and must be regenerated. Operators pick the floor per environment.
     </div>
   </div>
 </section>
@@ -366,7 +371,7 @@ <h2>Policy from one flag</h2>
 <span class="c"># Constrain the CLI version that produced the bundle.</span>
 <span class="p">$</span> aicr verify ./my-bundle --cli-version-constraint ">= 1.0.0"
 
-<span class="c"># CI: keep the verify exit code (a failed binary attestation reports attested but exits nonzero)</span>
+<span class="c"># CI: keep the verify exit code; failed integrity or attestation reports unknown</span>
 <span class="p">$</span> set -o pipefail; aicr verify ./my-bundle --format json | jq '.trustLevel, .trustReason'</pre>
     </div>
     <div class="grid2">
@@ -394,9 +399,10 @@ <h2>Tamper-evident</h2>
 <span class="r">Bundle verification: FAILED (non-zero exit)</span></pre>
     </div>
     <p>The signature&rsquo;s subject is the digest of <code class="inl">checksums.txt</code>, and
-      <code class="inl">checksums.txt</code> pins every file it lists. Mutating a listed file
-      breaks the checksum; mutating <code class="inl">checksums.txt</code> to match breaks the
-      signature.</p>
+      <code class="inl">checksums.txt</code> pins every regular payload file. Mutating a listed
+      file breaks the checksum; mutating or reordering <code class="inl">checksums.txt</code>
+      breaks the signature. Adding any unlisted file, directory, symlink, or other
+      non-regular object fails exact-tree verification.</p>
   </div>
 </section>
 

demos/bundle-attestation-demo.sh

@@ -20,9 +20,11 @@
 #   Verify  — aicr verify (with default + min-trust-level + tamper paths)
 #
 # When --attest is passed, the CLI signs `checksums.txt` with SLSA Build
-# Provenance v1 via Sigstore keyless OIDC (Fulcio cert + Rekor log entry),
-# and copies the binary's own SLSA attestation into the bundle so the
-# verifier can walk the full chain back to NVIDIA CI.
+# Provenance v1 via Sigstore keyless OIDC (Fulcio cert + Rekor log entry).
+# That manifest inventories every regular payload file, including recipe.yaml;
+# closed-world verification also rejects any additional filesystem entry. The
+# CLI copies its own SLSA attestation into the bundle so the verifier can walk
+# the full chain back to NVIDIA CI.
 #
 # Signing requires Fulcio + Rekor egress and an OIDC source — either ambient
 # GitHub Actions OIDC, or an interactive browser sign-in. On hosts where
@@ -201,7 +203,8 @@ fi   # end SKIP_ATTEST guard
 # --- inspect the bundle -------------------------------------------------------
 
 banner "Inspect the bundle layout"
-note "Content files + checksums.txt + attestation/ holding the two signed predicates."
+note "Closed-world payload inventory + checksums.txt + the two permitted attestation files."
+note "Only checksums.txt and the two attestation JSON files may sit outside the manifest."
 run ls -R "$BUNDLE"
 
 banner "Inspect the bundle attestation predicate"
@@ -214,8 +217,10 @@ run bash -c "jq '{ subject_count: (.dsseEnvelope.payload | @base64d | fromjson |
 # --- verify -------------------------------------------------------------------
 
 banner "Verify — default (auto-detect maximum trust level)"
-note "Five gates: checksums → bundle signature → bundle predicate → binary attestation chain → identity pin."
-note "The reported trust level is the highest level every gate passed at."
+note "Five gates: closed-world inventory → bundle signature → bundle predicate → binary attestation chain → identity pin."
+note "Extra files, directories, symlinks, or other non-regular objects fail verification."
+note "Legacy bundles with incomplete manifests report unknown trust and must be regenerated."
+note "The trust level reflects the verified chain and is capped at attested when external data was used."
 pause "Press Enter to run aicr verify (default)"
 run "$AICR" verify "$BUNDLE"
 
@@ -233,8 +238,9 @@ run bash -c "set -o pipefail; '$AICR' verify '$BUNDLE' --format json | jq '{ tru
 # --- tamper -------------------------------------------------------------------
 
 banner "Tamper-evident: mutate a content file, verify fails"
-note "The signature's subject is the digest of checksums.txt, which pins every generated payload file, including recipe.yaml."
-note "Mutating any file listed in checksums.txt breaks its checksum; mutating checksums.txt breaks the signature."
+note "The signature's subject is the digest of checksums.txt, which pins the complete payload inventory, including recipe.yaml."
+note "Mutating a listed file breaks its checksum; mutating or reordering checksums.txt breaks the signature."
+note "Adding an unlisted filesystem entry also fails the exact-tree check."
 # Pick a file the bundle is guaranteed to contain. README.md is always present.
 TAMPER_TARGET="$BUNDLE/README.md"
 if [ ! -f "$TAMPER_TARGET" ]; then

demos/bundle-attestation.md

@@ -2,13 +2,14 @@
 
 Bundle attestation provides cryptographic proof of **who** created a deployment
 bundle and **which AICR CLI** built it. When `aicr bundle --attest` runs, the
-CLI signs the bundle's `checksums.txt` (which inventories the deployment
-payload, including `recipe.yaml`; the attestation files are verified separately)
-using [Sigstore](https://www.sigstore.dev/) and
-generates SLSA Build Provenance v1 metadata. Anyone can later verify the
-bundle with `aicr verify` to confirm:
-
-* The generated payload files listed in `checksums.txt`, including `recipe.yaml`, haven't been tampered with.
+CLI signs the bundle's `checksums.txt` using
+[Sigstore](https://www.sigstore.dev/) and generates SLSA Build Provenance v1
+metadata. The manifest is a closed-world inventory of the deployment payload,
+including `recipe.yaml`; the attestation files are verified separately as part
+of the same inventory. Anyone can later verify the bundle with `aicr verify` to
+confirm:
+
+* Every required payload file is present and unchanged, and no additional filesystem entry is present.
 * It was created by a trusted identity.
 * It was built by an attested NVIDIA-CI-released AICR CLI.
 
@@ -88,8 +89,8 @@ permissions.
 
 ```text
 my-bundle/
-├── checksums.txt                          # SHA256 of every generated payload file
-├── recipe.yaml                            # canonical post-resolution recipe
+├── checksums.txt                          # closed-world SHA256 payload inventory
+├── recipe.yaml                            # canonical recipe; covered by checksums.txt
 ├── deploy.sh                              # automation script
 ├── README.md                              # deployment guide
 ├── 001-<component>/                       # per-component folder (NNN-prefixed)
@@ -101,13 +102,19 @@ my-bundle/
     └── aicr-attestation.sigstore.json     # binary SLSA attestation (copied in)
 ```
 
+`checksums.txt` covers every regular payload file shown above; the two
+attestation files are separately verified metadata outside the manifest. Any
+other unlisted file, directory, symlink, or special object fails verification.
+See [Artifact Verification](../docs/user/artifact-verification.md#what-can-be-verified)
+for the complete inventory contract and manifest-order rules.
+
 The two attestations together form the chain that makes `verified` reachable:
 
 * **`bundle-attestation.sigstore.json`** — Sigstore Bundle (DSSE + Fulcio cert
   + Rekor inclusion proof). Its in-toto subject is the SHA256 of
-  `checksums.txt`, so signing this one file transitively pins every generated
-  payload file that `checksums.txt` lists, including `recipe.yaml`. The signer
-  identity is the creator's OIDC identity.
+  `checksums.txt`, so signing this one file transitively pins the complete
+  closed-world payload inventory, including `recipe.yaml`. The signer identity
+  is the creator's OIDC identity.
 * **`aicr-attestation.sigstore.json`** — the SLSA Build Provenance attestation
   *of the AICR CLI binary that produced the bundle*, copied in at bundle time.
   Its signer identity is NVIDIA CI (`https://github.com/NVIDIA/aicr/.github/workflows/on-tag.yaml@...`).
@@ -142,13 +149,15 @@ Bundle verification: PASSED
 
 Five gates run, top to bottom:
 
-1. **Checksums** — every generated payload file listed in `checksums.txt`, including `recipe.yaml`, is hashed and compared.
+1. **Closed-world inventory** — every regular payload file, including `recipe.yaml`, is hashed and compared; additional files, directories, symlinks, and other non-regular objects are rejected.
 2. **Bundle signature** — the Sigstore Bundle is verified against the trusted root.
 3. **Bundle predicate** — the in-toto subject is checked against the actual `checksums.txt` digest.
 4. **Binary attestation chain** — `aicr-attestation.sigstore.json` is verified and its subject is checked against the CLI binary digest claimed in the bundle predicate.
 5. **Identity pin** — the binary attestation's signer is pinned to NVIDIA CI workflows.
 
-Any gate failing short-circuits to a lower trust level (see table below).
+An invalid inventory or failed attestation reports `unknown` trust. An absent
+attestation can still reach `unverified` when the closed-world inventory is
+valid (see the table below).
 
 ## 6. Policy enforcement
 
@@ -161,10 +170,13 @@ aicr verify ./my-bundle --min-trust-level attested
 
 | Level | Name | Criteria |
 |-------|------|----------|
-| **4** | `verified` | Checksums + bundle attestation + binary attestation pinned to NVIDIA CI |
-| **3** | `attested` | Bundle attestation verified; binary attestation missing/unverified, or external data used. A *failed* binary attestation also reports attested but exits nonzero (#1550) |
-| **2** | `unverified` | Checksums valid, `--attest` was not used |
-| **1** | `unknown` | Missing/invalid `checksums.txt`, or bundle attestation fails verification |
+| **4** | `verified` | Closed-world inventory + bundle attestation + binary attestation pinned to NVIDIA CI |
+| **3** | `attested` | Closed-world inventory + bundle attestation verified; binary attestation missing, or external data used |
+| **2** | `unverified` | Closed-world checksum inventory valid; `--attest` was not used |
+| **1** | `unknown` | Missing, invalid, or incomplete `checksums.txt`; unexpected filesystem entries; or bundle/binary attestation verification failure |
+
+Legacy bundles with incomplete manifests report `unknown` trust and must be
+regenerated before deployment.
 
 Pick the floor per environment. Production bundles must be `verified`; an
 emergency hotfix bundle built off-CI might only be required to reach
@@ -207,8 +219,8 @@ aicr verify ./my-bundle --format json | jq '{ trustLevel, bundleCreator, toolVer
 Branching in a pipeline:
 
 ```shell
-# Capture the JSON and the verify exit code separately: a binary-attestation
-# that *fails* verification still reports trustLevel=attested but exits nonzero.
+# Capture the JSON and the verify exit code separately: failed integrity or
+# attestation reports trustLevel=unknown and exits nonzero.
 if out=$(aicr verify ./my-bundle --format json); then rc=0; else rc=$?; fi
 trust=$(printf '%s' "$out" | jq -r .trustLevel)
 if [ "$rc" -ne 0 ]; then echo "fail — verify exited $rc" ; exit 1 ; fi
@@ -221,8 +233,8 @@ esac
 
 ## 8. Tamper demo
 
-The signed manifest hash pins every file listed in `checksums.txt`. Mutating
-a listed file breaks verification:
+The signed manifest hash pins every regular payload file in the closed-world
+inventory. Mutating a listed file breaks verification:
 
 ```shell
 # Component dirs are numbered NNN-<component>/; tamper the first one's values
@@ -241,6 +253,10 @@ Editing `checksums.txt` to match the new hash defeats the checksum gate but
 breaks the bundle signature gate — the signed subject is the digest of
 `checksums.txt` itself, which now doesn't match the signed value.
 
+Adding an unlisted file, directory, symlink, or other non-regular object also
+fails `aicr verify`: exact-tree validation rejects entries outside the verified
+inventory before any deployment gate can pass.
+
 ## Troubleshooting
 
 **"sigstore verification failed — trusted root may be stale"** — Sigstore