Close PRD gaps: custom types, bundle signatures, determinism, and gait fixtures

Author: davidahmann

.github/workflows/determinism.yml

@@ -0,0 +1,21 @@
+name: determinism
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  vectors:
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-go@v5
+        with:
+          go-version-file: go.mod
+      - name: Determinism vectors
+        run: go test ./core/signing -run TestDeterministicVector -count=1 -v
+

IMPLEMENTATION_CHECK.md

@@ -16,11 +16,11 @@ Status key:
 | ID | Status | Notes |
 |---|---|---|
 | FR1 Record creation | PASS | Deterministic record creation + validation in `core/record` + `proof.NewRecord()`. |
-| FR2 Type registry | PASS | Built-in registry + schema validation + `proof types list/validate`. |
+| FR2 Type registry | PASS | Built-in registry + runtime custom type registration + `proof types list/validate`. |
 | FR3 Hash chain | PASS | Append + verify + range verify + break-point reporting in `core/chain` and CLI. |
-| FR4 Signing | PASS | Ed25519 + cosign signing/verification paths implemented, including cert/identity/issuer verify options and revocation-list verification. |
-| FR5 Canonicalization | PASS | JSON/SQL/URL/text/prompt canonicalization in `core/canon`. |
-| FR6 Verification CLI | PASS | `verify`, `inspect`, `chain verify`, `types`, `frameworks`; exit code contract and JSON output implemented. |
+| FR4 Signing | PASS | Ed25519 + cosign signing/verification paths implemented for records/chains and bundle manifests, including cert/identity/issuer verify options and revocation-list verification. |
+| FR5 Canonicalization | PASS | JSON/SQL/URL/text/prompt canonicalization plus digest metadata (`algo_id`, `salt_id`) and HMAC-SHA-256 helpers in `core/canon`. |
+| FR6 Verification CLI | PASS | `verify`, `inspect`, `chain verify`, `types`, `frameworks`; bundle signature verification, custom type schema mapping, `--explain`, and exit code contract implemented. |
 | FR7 Framework definitions | PASS | 8 frameworks in `frameworks/` and `core/framework/`; list/show implemented. |
 | FR8 Go module API | PASS | Primary API surface exported from `proof.go`. |
 | FR9 JSON schemas | PASS | Base + type schemas + chain/bundle/framework schemas in `schemas/v1/`. |
@@ -35,11 +35,11 @@ Status key:
 | AC4 Cross-product chain | PASS | Mixed record types chain and verify correctly. |
 | AC5 Offline guarantee | PASS | Core verification is offline-first; cosign path is explicitly local-binary based, no mandatory network dependency in CLI flow. |
 | AC6 Schema validation | PASS | Invalid/missing fields rejected by schema and validation layers. |
-| AC7 Custom type | PASS | Custom schema validation is supported through CLI/API (`types validate`, `ValidateCustomTypeSchema`). |
+| AC7 Custom type | PASS | Runtime custom type registration is supported through CLI/API (`--custom-type-schema`, `RegisterCustomTypeSchema`), and verification validates base + custom schema. |
 | AC8 Framework PR only | PASS | Frameworks are YAML-only; no code change required to add files. |
-| AC9 Sigstore parity | PASS | cosign key/cert verification paths, release signing, and release signature verification are wired and tested. |
-| AC10 Determinism proof | PASS | Determinism/contract checks are gated (hash-chain integrity, exit-code contract, golden-style deterministic checks in tests/scripts). |
-| AC11 Gait backward compatibility | PASS | Native Gait pack and embedded signed-JSON verification with key-id compatibility and signature checks implemented and covered. |
+| AC9 Sigstore parity | PASS | cosign key/cert verification paths cover records/chains and Gait `proof_records.jsonl` verification with Sigstore options. |
+| AC10 Determinism proof | PASS | Deterministic vector assertions are enforced in tests with a dedicated cross-platform determinism workflow. |
+| AC11 Gait backward compatibility | PASS | Native Gait pack + signed-JSON verification with key-id compatibility is covered, including committed compatibility fixtures in `testdata/gait_compat/`. |
 | AC12 Exit code contract | PASS | Implemented and validated in unit + contract script. |
 
 ## Clyra_DEV Standards Check
@@ -51,7 +51,7 @@ Status key:
 | Pre-commit hooks | PASS | `.pre-commit-config.yaml` added. |
 | Testing tiers | PASS | Tiered scripts and workflows are present for unit/integration/e2e/acceptance/hardening/chaos/performance/soak/contract. |
 | Coverage gates | PASS | Coverage gates enforce package-level `>=85` for core/cmd stack (with narrow allowlist exceptions) and `>=75` package baseline. |
-| Main CI pipeline | PASS | PR and main workflows with lint/test/build/contract checks. |
+| Main CI pipeline | PASS | PR and main workflows with lint/test/build/contract checks, plus deterministic vector checks in cross-platform CI. |
 | Nightly pipelines | PASS | Nightly workflow includes hardening/chaos/soak/acceptance with cross-platform hardening matrix + performance job. |
 | Release integrity | PASS | GoReleaser, checksums, SBOM, grype, cosign sign/verify, and provenance attestation are wired in release workflow. |
 | Security scanning | PASS | CodeQL + `gosec` + `govulncheck` + release vulnerability scan are present in CI/release gates. |

README.md

@@ -12,6 +12,8 @@ Core capabilities:
 - Hash-chain append and integrity verification
 - Ed25519 signing and verification
 - Cosign/Sigstore-backed signature verification support
+- Bundle manifest signing and verification (Ed25519 + cosign)
+- Runtime custom record type registration (schema-per-type)
 - Offline CLI verification and inspection
 - Versioned schemas for proof record types
 - Gait pack/runpack compatibility verification
@@ -89,9 +91,10 @@ cosign verify-blob \
 ## What You Get
 
 - **Deterministic canonicalization**: JSON/SQL/URL/text domains with stable digests.
+- **Digest metadata**: `algo_id` + optional `salt_id`, including HMAC-SHA-256 helpers.
 - **Tamper evidence**: record hashes + chain head continuity checks.
 - **Signature options**: Ed25519 and cosign verification support.
-- **Schema contracts**: built-in proof record schemas and custom schema validation.
+- **Schema contracts**: built-in proof record schemas plus runtime custom type validation.
 - **Cross-product compatibility**: Gait pack/runpack verification and migration-friendly compatibility packages.
 
 ## CLI Surface
@@ -100,6 +103,7 @@ cosign verify-blob \
 proof verify <path>                               Verify record, chain, bundle, gait pack/runpack/signed JSON
 proof verify --signatures --public-key <key> <path>
 proof verify --signatures --cosign-key <pubkey-path> <path>
+proof verify --custom-type-schema <type>=<schema.json> <path>
 proof verify --revocation-list <path> --revocation-key <pubkey> <path>
 proof chain verify <path> [--from RFC3339] [--to RFC3339]
 proof inspect <path> [--record <record_id>]
@@ -114,7 +118,7 @@ Global flags:
 
 - `--json`
 - `--quiet`
-- `--explain`
+- `--explain` (step diagnostics to stderr)
 
 ## Gait Compatibility
 
@@ -133,6 +137,12 @@ For Gait extraction/migration support, Proof exposes compatibility packages:
 - `github.com/Clyra-AI/proof/schema`
 - `github.com/Clyra-AI/proof/exitcode`
 
+Compatibility fixtures used by tests:
+
+- `testdata/gait_compat/trace_signed.json`
+- `testdata/gait_compat/approval_token_signed.json`
+- `testdata/gait_compat/delegation_token_signed.json`
+
 ## Library Usage
 
 Primary API:
@@ -159,6 +169,19 @@ _, _ = proof.Sign(&chain.Records[0], key)
 _, _ = proof.VerifyChain(chain)
 ```
 
+Custom type registration + bundle signing:
+
+```go
+_ = proof.RegisterCustomTypeSchema("vendor.custom_event", "./custom.schema.json")
+
+manifest, _ := proof.SignBundle("./bundle", key)
+_, _ = proof.VerifyBundle("./bundle", proof.BundleVerifyOpts{
+  VerifySignatures: true,
+  PublicKey:        proof.PublicKey{Public: key.Public},
+})
+_ = manifest
+```
+
 ## Contract Commitments
 
 - Deterministic canonicalization and hashing for supported domains
@@ -182,6 +205,7 @@ Exit codes:
 
 ![Main](https://github.com/Clyra-AI/proof/actions/workflows/main.yml/badge.svg)
 ![CodeQL](https://github.com/Clyra-AI/proof/actions/workflows/codeql.yml/badge.svg)
+![Determinism](https://github.com/Clyra-AI/proof/actions/workflows/determinism.yml/badge.svg)
 
 ```bash
 make fmt
@@ -195,6 +219,7 @@ Key automation:
 
 - Main CI: `.github/workflows/main.yml`
 - PR CI: `.github/workflows/pr.yml`
+- Determinism CI: `.github/workflows/determinism.yml`
 - Nightly hardening/perf: `.github/workflows/nightly.yml`
 - Release: `.github/workflows/release.yml`
 

cmd/proof/artifacts.go

@@ -3,8 +3,6 @@ package main
 import (
 	"archive/zip"
 	"bufio"
-	"crypto/sha256"
-	"encoding/hex"
 	"encoding/json"
 	"errors"
 	"fmt"
@@ -30,15 +28,6 @@ const (
 	artifactGaitSignedJSON artifactKind = "gait_signed_json"
 )
 
-type manifest struct {
-	Files []manifestEntry `json:"files"`
-}
-
-type manifestEntry struct {
-	Path   string `json:"path"`
-	SHA256 string `json:"sha256"`
-}
-
 func detectArtifact(path string) (artifactKind, error) {
 	info, err := os.Stat(path)
 	if err != nil {
@@ -150,30 +139,21 @@ func loadChain(path string) (*proof.Chain, error) {
 	return c, nil
 }
 
-func verifyBundle(path string) error {
-	// #nosec G304 -- CLI accepts explicit local artifact paths.
-	raw, err := os.ReadFile(filepath.Join(path, "manifest.json"))
-	if err != nil {
-		return err
-	}
-	var m manifest
-	if err := json.Unmarshal(raw, &m); err != nil {
-		return err
-	}
-	for _, file := range m.Files {
-		// #nosec G304 -- Bundle manifest drives local file verification.
-		b, err := os.ReadFile(filepath.Join(path, file.Path))
+func verifyBundle(path string, verifySignatures bool, publicKey string, cosignOpts proof.CosignVerifyOpts) error {
+	var pub proof.PublicKey
+	if strings.TrimSpace(publicKey) != "" {
+		decoded, err := decodePublicKey(publicKey)
 		if err != nil {
 			return err
 		}
-		sum := sha256.Sum256(b)
-		got := hex.EncodeToString(sum[:])
-		want := strings.TrimPrefix(file.SHA256, "sha256:")
-		if got != want {
-			return fmt.Errorf("bundle hash mismatch for %s", file.Path)
-		}
+		pub = decoded
 	}
-	return nil
+	_, err := proof.VerifyBundle(path, proof.BundleVerifyOpts{
+		VerifySignatures: verifySignatures,
+		PublicKey:        pub,
+		Cosign:           cosignOpts,
+	})
+	return err
 }
 
 func appendJSONLRecords(c *proof.Chain, path string) error {
@@ -201,22 +181,28 @@ func appendJSONLRecords(c *proof.Chain, path string) error {
 	return scanner.Err()
 }
 
-func verifyGaitPack(path string, verifySignatures bool, publicKey string) (*gait.Result, error) {
+func verifyGaitPack(path string, verifySignatures bool, publicKey string, cosignOpts proof.CosignVerifyOpts) (*gait.Result, error) {
 	var pubKey []byte
 	if verifySignatures {
-		if strings.TrimSpace(publicKey) == "" {
+		if strings.TrimSpace(publicKey) != "" {
+			pub, err := decodePublicKeyValue(publicKey)
+			if err != nil {
+				return nil, err
+			}
+			pubKey = pub
+		}
+		if len(pubKey) == 0 && strings.TrimSpace(cosignOpts.KeyPath) == "" && strings.TrimSpace(cosignOpts.CertificatePath) == "" {
 			return nil, fmt.Errorf("--public-key is required for gait pack signature verification")
 		}
-		pub, err := decodePublicKeyValue(publicKey)
-		if err != nil {
-			return nil, err
-		}
-		pubKey = pub
 	}
-	return gait.VerifyPack(path, verifySignatures, pubKey)
+	return gait.VerifyPackWithOptions(path, gait.VerifyOpts{
+		VerifySignatures: verifySignatures,
+		PublicKey:        pubKey,
+		Cosign:           cosignOpts,
+	})
 }
 
-func verifyGaitRunpack(path string, verifySignatures bool, publicKey string) (*gait.RunpackResult, error) {
+func verifyGaitRunpack(path string, verifySignatures bool, publicKey string, _ proof.CosignVerifyOpts) (*gait.RunpackResult, error) {
 	var pubKey []byte
 	if verifySignatures {
 		if strings.TrimSpace(publicKey) == "" {
@@ -228,7 +214,10 @@ func verifyGaitRunpack(path string, verifySignatures bool, publicKey string) (*g
 		}
 		pubKey = pub
 	}
-	return gait.VerifyRunpack(path, verifySignatures, pubKey)
+	return gait.VerifyRunpackWithOptions(path, gait.VerifyOpts{
+		VerifySignatures: verifySignatures,
+		PublicKey:        pubKey,
+	})
 }
 
 func verifyGaitSignedJSON(path, publicKey string) error {

cmd/proof/artifacts_test.go

@@ -66,7 +66,7 @@ func TestVerifyBundle(t *testing.T) {
 	manifestJSON := `{"files":[{"path":"records.jsonl","sha256":"sha256:ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356"}]}`
 	testutil.WriteFile(t, filepath.Join(dir, "manifest.json"), []byte(manifestJSON))
 
-	require.NoError(t, verifyBundle(dir))
+	require.NoError(t, verifyBundle(dir, false, "", proof.CosignVerifyOpts{}))
 }
 
 func TestDecodePublicKeyErrors(t *testing.T) {
@@ -144,7 +144,7 @@ func TestDetectAndVerifyGaitPack(t *testing.T) {
 	require.NoError(t, err)
 	require.Equal(t, artifactGaitPack, kind)
 
-	res, err := verifyGaitPack(zipPath, true, hex.EncodeToString(pub))
+	res, err := verifyGaitPack(zipPath, true, hex.EncodeToString(pub), proof.CosignVerifyOpts{})
 	require.NoError(t, err)
 	require.Equal(t, 2, res.FilesVerified)
 	require.Equal(t, 1, res.ProofRecordsVerified)
@@ -224,7 +224,7 @@ func TestDetectAndVerifyGaitRunpack(t *testing.T) {
 	require.NoError(t, err)
 	require.Equal(t, artifactGaitRunpack, kind)
 
-	res, err := verifyGaitRunpack(zipPath, true, base64.StdEncoding.EncodeToString(pub))
+	res, err := verifyGaitRunpack(zipPath, true, base64.StdEncoding.EncodeToString(pub), proof.CosignVerifyOpts{})
 	require.NoError(t, err)
 	require.Equal(t, "run-1", res.RunID)
 	require.Equal(t, 4, res.FilesVerified)

cmd/proof/chain.go

@@ -17,6 +17,7 @@ func newChainCmd(opts *globalOpts) *cobra.Command {
 		Short: "Verify chain integrity",
 		Args:  cobra.ExactArgs(1),
 		RunE: func(cmd *cobra.Command, args []string) error {
+			explainf(opts, "chain verify path=%s from=%s to=%s", args[0], fromStr, toStr)
 			c, err := loadChain(args[0])
 			if err != nil {
 				return newCLIError(exitcode.InvalidInput, err.Error())

cmd/proof/explain.go

@@ -0,0 +1,13 @@
+package main
+
+import (
+	"fmt"
+	"os"
+)
+
+func explainf(opts *globalOpts, format string, args ...any) {
+	if opts == nil || !opts.explain || opts.quiet {
+		return
+	}
+	_, _ = fmt.Fprintf(os.Stderr, "explain: "+format+"\n", args...)
+}

cmd/proof/frameworks.go

@@ -14,6 +14,7 @@ func newFrameworksCmd(opts *globalOpts) *cobra.Command {
 		Use:   "list",
 		Short: "List available frameworks",
 		RunE: func(cmd *cobra.Command, args []string) error {
+			explainf(opts, "frameworks list")
 			list, err := framework.List()
 			if err != nil {
 				return newCLIError(exitcode.InternalError, err.Error())
@@ -33,6 +34,7 @@ func newFrameworksCmd(opts *globalOpts) *cobra.Command {
 		Short: "Show a framework definition",
 		Args:  cobra.ExactArgs(1),
 		RunE: func(cmd *cobra.Command, args []string) error {
+			explainf(opts, "frameworks show id=%s", args[0])
 			f, err := framework.Load(args[0])
 			if err != nil {
 				return newCLIError(exitcode.InvalidInput, err.Error())