Skip to content

Latest commit

 

History

History
595 lines (419 loc) · 25.2 KB

File metadata and controls

595 lines (419 loc) · 25.2 KB

VSC — Versioned State Commit

VSC Conformance

v1.18 — Evidence Flow Demo

Research prototype / proof-of-concept. Not production-ready.

License: VSC Core is licensed under the Apache License 2.0. The names DigiEmu and VSC, logos, and project branding are not granted as trademarks.

Non-technical overview

For plain-language explanations of what VSC proves, what it does not prove, and how to read PASS / FAIL / ERROR results:


VSC v2.0 Formal Specification Draft

The VSC v2.0 formal specification draft is available in docs/vsc-v2-0-formal-specification-draft.md.

It defines the VSC evidence model, token roles, bundle structure, verifier behavior, and validation semantics in a language-independent form. v1.x remains the current Node.js reference implementation.

The VSC v2.1 Canonical Event Model draft is available in docs/vsc-v2-1-canonical-event-model-draft.md. It defines the smallest verifiable event unit used by VSC evidence flows.

The VSC v2.2 Conformance Test Vectors draft is available in docs/vsc-v2-2-conformance-test-vectors-draft.md. It defines the compatibility test surface for future VSC implementations.

The VSC v2.3 Interop Receipt Profile draft is available in docs/vsc-v2-3-interop-receipt-profile-draft.md. It defines how external systems may reference VSC evidence while preserving VSC result semantics and evidence boundaries.

The VSC v2.4 Go Core Prototype Preparation draft is available in docs/vsc-v2-4-go-core-prototype-preparation.md. It defines the preparation path for a future Go-based VSC verifier while preserving the Node.js v1.x reference implementation.

Experimental Go verifier prototype (v2.6.1): go run ./cmd/vsc-go verify-bundle <bundle-folder>. Read-only. Preserves VSC result semantics. Node.js remains the reference implementation. JSON output: go run ./cmd/vsc-go verify-bundle --json <bundle-folder> (follows the VSC v2.6 Machine-readable Verification Result Schema draft).

The VSC v2.6 Machine-readable Verification Result Schema draft is available in docs/vsc-v2-6-machine-readable-verification-result-schema-draft.md. It defines a stable JSON result surface for future Go --json output, conformance comparisons, APIs, and interop receipts.

The VSC v2.7 Conformance Fixture Package is available in conformance/v2.7/. It provides minimal PASS / FAIL / ERROR fixtures for comparing verifier behavior across implementations.

The VSC v2.8 Node/Go Result Comparison Draft is available in conformance/v2.8/. It defines how verifier implementations should compare result classes across the v2.7 conformance fixtures. The v2.8.2 comparison runner supports human-readable and machine-readable output: npm run vsc -- compare:fixtures (human-readable table) or npm run vsc -- compare:fixtures --json (JSON, intended for automation and future CI).

Interop / boundary notes

VSC can be discussed as part of a broader trust architecture with identity / trust and responsibility / attribution layers.

docs/interop/TBN_VSC_CLARIXO_BOUNDARIES.md — TBN / VSC / CLARIXO safe interoperability boundaries. This document describes reference patterns. It is not an integration-complete claim.

External reference profile draft: docs/interop/EXTERNAL_REFERENCE_PROFILE_DRAFT.md and schemas/external-reference-profile-v2.18-draft.schema.json. External references are pointers, not proof imports.

External reference examples (draft, not production integrations): examples/external-references/

External reference validation notes: docs/interop/EXTERNAL_REFERENCE_VALIDATION_NOTES.md. Reference validation is not artifact validation.

Interop roadmap / pre-v3 planning: docs/interop/INTEROP_ROADMAP_PRE_V3.md. Planning only; does not start v3.0 or implement external reference validation.

External reference shape validation plan: docs/interop/EXTERNAL_REFERENCE_SHAPE_VALIDATION_PLAN.md. Planning only. Shape validation checks the reference object, not the external artifact.

External reference fixture plan: docs/interop/EXTERNAL_REFERENCE_FIXTURE_PLAN.md. Fixture planning does not implement fixture validation.

Interop consolidation summary: docs/interop/INTEROP_CONSOLIDATION_SUMMARY.md. Consolidates v2.17–v2.23. Documentation only; does not implement validation or start v3.0.

Partner / reviewer interop brief: docs/interop/PARTNER_REVIEWER_INTEROP_BRIEF.md. Compact partner/reviewer entry point. Documentation only; does not implement validation or start v3.0.

Enterprise review package: docs/review/ENTERPRISE_REVIEW_PACKAGE.md. Review package only; not certification. Includes validation commands, expected outputs, key files, and explicit non-claims.

Partner-facing review brief: docs/review/PARTNER_REVIEW_BRIEF.md. Short PDF/source-friendly entry point; not certification.

CI Conformance Check

VSC includes a GitHub Actions workflow for automated conformance validation.

The workflow runs on every push and pull request targeting main and checks:

npm run vsc -- compare:fixtures
npm run vsc -- compare:fixtures --json
npm run vsc -- verify-all

The JSON comparison output must be valid JSON and must report:

{
  "result": "COMPARE_PASS"
}

This CI check validates fixture-level comparison behavior and full repository verification status. It does not change VSC verification semantics.

CI artifacts

Successful CI conformance runs publish downloadable workflow artifacts:

comparison-result.json
comparison-report.md

comparison-result.json is the machine-readable comparison result generated by npm run vsc -- compare:fixtures --json. comparison-report.md is a small human-readable summary of the same CI comparison result. These artifacts make CI conformance results easier to inspect and archive.

Artifacts are uploaded with run-aware and commit-aware names:

vsc-comparison-artifacts-run-<run_number>-<commit_sha>

Artifacts are retained for 30 days by the GitHub Actions workflow.


Core Concept

VSC is a base + delta + proof + recovery model for folder and file state:

T = (B, Δ, P, R)

B = base snapshot     — full state, chunked and root-hashed
Δ = ordered deltas    — only the files that changed
P = proof             — root hash / chain hash for verification
R = recovery          — rules for reconstructing state from base + deltas

Store a full state once. Store only what changes. Verify by root hash. Restore from base + ordered deltas.


WordPress MVP Demo

A WordPress-style folder was backed up once (1.58 MB base), then two later changes were captured as ordered deltas (641 B total). The latest state was reconstructed and verified.

Strategy Storage
Traditional full-copy (3×) 4.75 MB
VSC (base + 2 deltas) 1.58 MB
Saved 3.17 MB (66.65%)

verify-all result: PASS: 3 / PROOF-ONLY: 12 / FAIL: 0

Full demo details


Visual Token Language

VSC tokens are rendered as SVG seals. Each token type has a distinct visual identity:

Token Core symbol Reading
FOLDER ⊞ BASE SNAPSHOT Stable complete state
DELTA Δ CHANGED FILES Changed subset, BASE → TARGET
CHAIN N DELTA STEPS Ordered progression, latest verified
RECOVERY ▣ RECOVERABLE Dense chunked payload
PROOF ◌ SPARSE PROOF Lightweight deterministic proof

Full visual grammar


Current Status

What works:

  • Folder base snapshots, restore, verification
  • Folder deltas (file-level add / modify / delete)
  • Delta-on-delta and delta chains
  • Chain restore and chain verification
  • Chain storage report
  • Static token gallery with SVG seal previews
  • verify-all batch verification

Not yet built:

  • API layer
  • Distributed object storage
  • Authentication / tenant model
  • Production hardening

CLI Commands

# Encode folder base
npm run encode-folder <folder>

# Encode folder delta
npm run encode-folder-delta <base-token.json> <folder>

# Build delta chain
npm run create-chain <base.json> <delta1.json> [delta2.json ...]

# Restore latest state from chain
npm run restore-chain <chain.json>

# Verify restored state
npm run verify-chain <chain.json> <restored-folder>

# Storage report
npm run report-chain <chain.json>

# Verify all tokens
npm run verify-all

# Clean manifest
npm run clean-manifest

# Open gallery
start output\gallery.html

VSC v1.11 GitHub Pages Showcase

The static showcase is publicly deployed via GitHub Pages:

https://digiemu.github.io/vsc-core/

Only showcase/ is published. No recovery chunks, test fixtures, or heavy output folders are deployed. The deployment is triggered automatically on every push to main.

Publishing documentation


VSC v1.12 Benchmark Mode

VSC v1.12 adds Benchmark Mode — reproducible measurement of storage-load reduction and restore/verify performance.

npm run vsc -- benchmark        # default: medium profile (100 states)
npm run vsc -- benchmark small  # 10 states - fast smoke test
npm run vsc -- benchmark large  # 1000 states - extended test

What it measures:

  • Storage comparison: VSC base+delta vs traditional full-copy storage
  • Timing: Base snapshot, delta generation, chain creation, restore, verify
  • Verification: Restore integrity and root-hash verification pass/fail

Sample output:

  • Total chain reduction: 90-98% (profile-dependent)
  • Saved bytes: ~2-5 MB for medium fixture
  • Restore time: ~500-1500ms
  • Verify time: ~200-500ms

Benchmark Mode makes VSC's storage-load reduction claims measurable and reproducible without introducing unbenchmarked enterprise claims.

Note: Benchmark output is written to output/benchmark/ and is separate from the public showcase. The showcase (npm run vsc -- showcase) uses the stable WordPress-style demo, not benchmark artifacts.

Full benchmark documentation
VSC v2 Architecture Notes


VSC v1.13 JSON Event Benchmark

VSC v1.13 adds JSON Event Benchmark — extending storage-load reduction measurement to structured AI decision event logs represented as deterministic JSON.

npm run vsc -- benchmark:json        # default: medium profile (100 states)
npm run vsc -- benchmark:json small  # 10 states - fast smoke test
npm run vsc -- benchmark:json large  # 1000 states - extended test

Event Model:

  • Primary unit of proof: Individual Event (prompt/response, tool call, policy check, retrieval, final decision)
  • Deterministic JSON with canonical key ordering
  • Accumulating event log: each state contains all events from 0 to N
  • Events include: event_id, sequence, timestamp, event_type, actor, model, input_hash, output_hash, policy_result, tool_calls, metadata

What it measures:

  • Storage efficiency for growing AI event logs
  • VSC delta encoding performance on structured JSON
  • Restore and verify timing for event state chains

Note: This benchmark models AI-style events but uses synthetic deterministic data. It is a bridge toward future AI evidence logging, not a claim of enterprise-ready AI log infrastructure.

Separation: JSON Event Benchmark output is written to output/json-benchmark/ and is separate from both the public showcase and the v1.12 folder benchmark.

Gallery: The public showcase gallery uses an allowlist approach and does not include benchmark or JSON benchmark generated SVGs, keeping the GitHub Pages gallery deterministic and lightweight.

Full JSON Event Benchmark documentation


VSC v1.14 Evidence Bundle Export

VSC v1.14 adds Evidence Bundle Export — a CLI command that turns a verified VSC delta chain into a portable evidence package containing all proof artifacts.

npm run vsc -- bundle output/vsc-chain-21A8390BFA3F-to-954BEB0FF3AA.json

What's included:

  • Chain token (complete delta chain with all steps)
  • Base token (initial state snapshot)
  • Delta tokens (ordered state changes)
  • Chain report (human-readable summary)
  • SVG seals (visual proof representations)
  • Verification summary with expected hashes
  • Complete manifest
  • SHA-256 checksums for integrity

Bundle structure:

output/bundles/vsc-bundle-<BASE>-to-<LATEST>/
├── README.md                    # Bundle guide
├── manifest.json                # Complete artifact inventory
├── chain-token.json             # The delta chain
├── base-token.json              # Base state
├── verification-summary.json    # Verification instructions
├── checksums.sha256            # File integrity checksums
├── delta-tokens/delta-*.json   # Delta steps
├── reports/chain-report.md     # Human-readable report
└── seals/*.svg                 # Visual seals

Use cases:

  • Partner sharing — Send verifiable proof to collaborators
  • Audit trail — Create compliance evidence
  • Research publication — Package reproducible proof artifacts
  • DigiEmu proof — Export as proof artifact

Note: Bundles do not include heavy recovery chunk folders or restored state (these can be regenerated from the tokens).

Full Evidence Bundle Export documentation


VSC v1.15 JSON Event Evidence Bundle

VSC v1.15 adds JSON Event Evidence Bundle Export — a specialized export mode that turns deterministic JSON AI event chains into portable evidence packages for audit, post-incident review, and DigiEmu verification workflows.

# Export bundle using latest JSON benchmark results
npm run vsc -- bundle:json

# Export with explicit chain token
npm run vsc -- bundle:json output/json-benchmark/vsc-chain-*.json

Prerequisites:

npm run vsc -- benchmark:json   # must run first to generate JSON benchmark data

What's included:

  • Event schema (event-schema.json) — Event structure definition
  • Event summary (event-summary.json) — Session statistics
  • JSON benchmark summary, report, and chart data
  • Chain token, base token, delta tokens
  • Verification summary with restore/verify status
  • Complete manifest and SHA-256 checksums

Bundle structure:

output/json-event-bundles/vsc-json-event-bundle-<BASE>-to-<LATEST>/
├── README.md                           # Bundle guide
├── manifest.json                       # Complete inventory
├── event-schema.json                   # Event structure
├── event-summary.json                  # Session statistics
├── json-benchmark-summary.json         # Benchmark metrics
├── json-benchmark-report.md            # Human report
├── json-benchmark-chart-data.json      # Chart data
├── chain-token.json                    # Delta chain
├── base-token.json                     # Base state
├── verification-summary.json           # Verification status
├── checksums.sha256                   # Checksums
└── delta-tokens/delta-*.json          # Delta steps

Event Model:

  • Primary unit of proof: Individual Event
  • Event types: prompt_response, tool_call, policy_check, retrieval, final_decision
  • Required fields: event_id, sequence, timestamp, event_type, actor, session_id, model, input_hash, output_hash, policy_result, tool_calls, metadata
  • Determinism: Synthetic deterministic data for reproducibility

Use cases:

  • Audit trail — AI decision event logging for compliance
  • Post-incident review — Capture decision timeline for investigation
  • Research publication — Reproducible AI event evidence
  • DigiEmu proof — Portable AI event proof artifact

Limitations:

  • Synthetic deterministic events — not real AI inference
  • Single-session model — not multi-session orchestration
  • Simulated tool calls — not actual execution
  • Research prototype — not enterprise production software

Full JSON Event Evidence Bundle documentation


VSC v1.16 Evidence Bundle Verification

VSC v1.16 adds Evidence Bundle Verification — a read-only verifier that checks whether an exported evidence bundle is complete, internally consistent, and unchanged according to its manifest and checksums.

# Verify a generic evidence bundle
npm run vsc -- verify-bundle output/bundles/vsc-bundle-<BASE>-to-<LATEST>

# Verify a JSON event evidence bundle
npm run vsc -- verify-bundle output/json-event-bundles/vsc-json-event-bundle-<BASE>-to-<LATEST>

What it verifies:

  • Required files — README.md, manifest.json, checksums.sha256, chain-token.json, base-token.json, verification-summary.json
  • Checksum integrity — Every file in checksums.sha256 exists and matches its hash
  • Manifest consistency — Manifest references match chain/base/delta token IDs
  • Chain token — Valid JSON, contains base/latest token IDs and steps
  • Base token — Exists and is valid JSON
  • Delta tokens — All deltas referenced by the chain exist in delta-tokens/
  • JSON event metadata — (for JSON event bundles) event-schema.json, event-summary.json, json-benchmark-summary.json are valid

Expected output:

VSC v1.16 — Evidence Bundle Verification
...
Manifest:        PASS
Checksums:       PASS (XX files verified)
Chain token:     PASS (BASE → LATEST)
Base token:      PASS
Delta tokens:    PASS (XX/XX found)
JSON event meta: PASS or N/A
Result:          PASS

Use cases:

  • Pre-sharing validation — Verify bundle integrity before sending to partners
  • Receipt verification — Confirm received bundle is unchanged
  • Audit compliance — Prove bundle completeness for compliance review
  • CI/CD integration — Automated bundle verification in pipelines

Note: This command is read-only. It never modifies the bundle (no manifest rewrite, no checksum update, no token mutation).

Full Evidence Bundle Verification documentation


VSC v1.17 ZIP Bundle Export

VSC v1.17 adds ZIP Bundle Export — a command to package an existing evidence bundle directory into a shareable ZIP archive.

npm run vsc -- zip-bundle output\json-event-bundles\vsc-json-event-bundle-408C8C13C4D4-to-ED9566562A13

What it does:

  • Creates a .zip file from the bundle folder
  • Outputs to output/zips/<bundle-name>.zip
  • Preserves the bundle directory structure inside the ZIP
  • Does not modify the source bundle

Expected output:

VSC v1.17 — ZIP Bundle Export
...
ZIP path:        output\zips\vsc-json-event-bundle-408C8C13C4D4-to-ED9566562A13.zip
Files included:  108
ZIP size:        245.6 KB
Result:          PASS

Use cases:

  • Email distribution — Share bundles as single-file attachments
  • Archive storage — Compress bundles for long-term storage
  • Distribution packaging — Prepare bundles for download/release

Note: The ZIP file is created in output/zips/ and is not committed to Git (see .gitignore).

Full ZIP Bundle Export documentation


VSC v1.18 Evidence Flow Demo

VSC v1.18 adds Evidence Flow Demo — a single command that runs the complete VSC evidence handoff flow end-to-end.

npm run vsc -- demo:evidence-flow

What it does:

  • Exports the JSON event evidence bundle (bundle:json)
  • Verifies the exported bundle (verify-bundle)
  • Packages the bundle as a portable ZIP (zip-bundle)
  • Prints a clear final summary with PASS/FAIL

Expected output:

VSC v1.18 — Evidence Flow Demo

  Flow:  Export  →  Verify  →  ZIP  →  Summary
...
  Verification result:  PASS
  ZIP export result:    PASS
  Final result:         PASS

This command is an orchestrator. Evidence logic, checksum binding, manifest integrity, and ZIP creation remain in their respective dedicated scripts. Use verify-bundle and zip-bundle independently at any time.

Full Evidence Flow Demo documentation


VSC v1.10 Static Showcase

VSC v1.10 adds a static showcase export. One command generates a portable showcase/ folder with the SVG seals, demo metrics, verification summary, and gallery — no server required, opens in any browser.

npm run vsc -- demo:run   # run full proof flow first
npm run vsc -- showcase   # export showcase/

Full showcase documentation


VSC v1.9 One-command Demo

VSC v1.9 adds a complete reproducible demo command. One command runs the full WordPress-style base + delta + chain + restore + verify flow:

npm run vsc -- demo:run

No manual token ID replacement. No placeholder values. The runner discovers generated paths from the manifest and prints a final summary with DEMO RESULT: PASS.

Full demo documentation


VSC v1.8 CLI

VSC v1.8 introduces a simplified CLI wrapper. The old scripts remain available and unchanged.

npm run vsc -- help          # command overview
npm run vsc -- demo          # create demo fixture
npm run vsc -- backup test-wp
npm run vsc -- delta output\vsc-<BASE>-folder-recovery.json test-wp
npm run vsc -- chain <base> <delta1> <delta2>
npm run vsc -- restore output\vsc-chain-<BASE>-to-<D2>.json
npm run vsc -- verify  output\vsc-chain-<BASE>-to-<D2>.json output\chain-<BASE>-to-<D2>\restored-test-wp
npm run vsc -- report  output\vsc-chain-<BASE>-to-<D2>.json
npm run vsc -- verify-all
npm run vsc -- gallery

Full CLI documentation


Repository / Demo Notes

This repository is a research prototype / MVP. The following conventions apply:

  • Generated recovery chunk folders (output/recovery-*/) are excluded from git — they can be large and are always reproducible locally.
  • The test-wp/ fixture folder is excluded from git. Recreate it with npm run demo:fixture.
  • Token JSON and SVG files in output/ are lightweight and may be committed as reference artifacts.
  • See docs/demo-artifact-policy.md for the full artifact commit policy.
  • See docs/reproduce-wordpress-demo.md to reproduce the full demo from a clean clone.
npm run demo:fixture       # create or reset test-wp/ demo fixture
npm run encode-folder      # encode a folder as a base snapshot
npm run encode-folder-delta # encode a folder delta
npm run create-chain       # build a delta chain
npm run report-chain       # storage metrics report
npm run restore-chain      # restore latest state from chain
npm run verify-chain       # verify restored state
npm run verify-all         # verify all registered tokens
npm run clean-manifest     # remove stale manifest entries

Documentation