Merge pull request #19 from 0xSteph/2.0-trust-spine

ptai 1.0.0 — the Trust Spine

Author: 0xSteph

.github/workflows/ci.yml

@@ -41,9 +41,9 @@ concurrency:
 
 jobs:
   # Dedicated job to satisfy the branch-protection rule, which requires a
-  # status check literally named "pytest (Python 3.12)". The matrix-driven
-  # job below covers ubuntu+macos+windows × 3.10/3.11/3.13; ubuntu+3.12 is
-  # excluded there to avoid double-running, since this job already covers it.
+  # status check literally named "pytest (Python 3.12)". This file runs the
+  # full suite once on ubuntu + Python 3.12; there is no cross-OS/version
+  # matrix here, so this is the only unguarded pytest invocation to harden.
   pytest:
     name: pytest (Python 3.12)
     runs-on: ubuntu-latest
@@ -63,7 +63,17 @@ jobs:
 
       - name: Run tests
         # Engagement-lifecycle E2E is the e2e-juiceshop job's job; ignore here.
-        run: pytest tests/ -v --tb=short --ignore=tests/test_engagement_lifecycle_e2e.py
+        # rm -f first so a leftover report.xml from any earlier step can never
+        # satisfy the guard if this run dies before writing its own.
+        run: |
+          rm -f report.xml
+          pytest tests/ -v --tb=short --junitxml=report.xml --ignore=tests/test_engagement_lifecycle_e2e.py
+
+      - name: Verify pytest result is complete (guard against os._exit truncation)
+        # The load-bearing defense: an os._exit(0) mid-run kills pytest before it
+        # writes report.xml, so a green return code yields no report at all. The
+        # guard fails the build on a missing/truncated report or any failure/error.
+        run: python scripts/check_pytest_result.py report.xml
 
   lint:
     name: lint + type check (ubuntu / py3.12)

.github/workflows/release.yml

@@ -51,9 +51,18 @@ jobs:
           uv pip install -e ".[dev]"
 
       - name: Run tests
+        # rm -f first so a stale report.xml can never satisfy the guard, and
+        # emit junitxml so the guard can detect an os._exit truncation. A
+        # green-but-truncated release gate would ship a broken package.
         run: |
           source .venv/bin/activate
-          python -m pytest tests/ --tb=short -q
+          rm -f report.xml
+          python -m pytest tests/ --tb=short -q --junitxml=report.xml
+
+      - name: Verify pytest result is complete (guard against os._exit truncation)
+        run: |
+          source .venv/bin/activate
+          python scripts/check_pytest_result.py report.xml
 
   build:
     needs: test

CHANGELOG.md

@@ -2,6 +2,30 @@
 
 All notable changes to ptai are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/), versioning follows [Semantic Versioning](https://semver.org/).
 
+## [1.0.0] - 2026-06-25
+
+The Trust Spine release. ptai now proves what it finds: a finding earns the VERIFIED badge only when a machine oracle re-runs the exploit and reproduces it N out of N times. No oracle, no badge, and no verdict from an LLM assertion.
+
+### Added
+
+- **Verifier + receipt contract.** A verified verdict must name the machine oracle that earned it, enforced in code.
+- **Recipe-driven oracles:** unescaped reflection, open redirect, IDOR/BOLA differential, error signature, MCP exposure, SQLi boolean and boolean-blind, SSTI (including error-based), sensitive data exposure, and HTTP request smuggling (CL.TE), plus an out-of-band OAST callback oracle for blind SSRF/XXE over a self-hosted loopback collaborator.
+- **Portable proof capsules and `ptai replay`**, plus multi-hop chain capsules, so a finding's proof travels and replays without trusting ptai.
+- **Prove-or-kill gating:** third-party scanner output (nuclei, nikto, zap) is held back until an oracle re-proves it.
+- **Verified-only SARIF export** (`ptai export --sarif`) with a frozen, versioned export-properties contract.
+- **REST path-parameter injection** in the SQLi/XSS/SSTI fuzzers, so injection in `/rest/products/<id>` style routes is caught, not just query parameters.
+- **Experimental CL.TE request-smuggling discovery probe**, oracle-gated.
+- **CI gate** (`--fail-on verified`) with a composite action, and a bundled `ptai demo`.
+
+### Changed
+
+- **Honest verdicts.** An oracle miss now reads as `candidate` (could not re-prove), never `refuted`; `refuted` is reserved for an oracle that can truly disprove a vulnerability.
+- **Impact-honest severity:** a bare out-of-band callback proves existence, not impact, so it is rated medium until impact is reproduced.
+
+### Verified
+
+- 100% precision with zero false positives on the honeypot benchmark, and field-validated against live OWASP Juice Shop: a real broken-object-level-authorization bug verified end to end with a replayable proof capsule.
+
 ## [0.17.2] - 2026-06-09
 
 Patch release: tool-installer fixes from user feedback on issue #12 (`lukeswitz`).

README.md

@@ -4,7 +4,7 @@
 
 <h1>pentest-ai</h1>
 
-**Autonomous pentests from one command. Real tools, working PoCs, audit-ready reports.**
+**The pentest tool that proves its findings. No oracle, no badge.**
 
 [![PyPI](https://img.shields.io/pypi/v/ptai?color=red&label=pypi&style=flat-square)](https://pypi.org/project/ptai/)
 [![Python](https://img.shields.io/badge/python-3.10%2B-red?style=flat-square)](https://pypi.org/project/ptai/)
@@ -13,29 +13,47 @@
 [![Stars](https://img.shields.io/github/stars/0xSteph/pentest-ai?color=red&style=flat-square)](https://github.com/0xSteph/pentest-ai/stargazers)
 [![Discord](https://img.shields.io/badge/discord-join-red?style=flat-square&logo=discord&logoColor=white)](https://discord.gg/6weeTAubJw)
 
-[**Website**](https://pentestai.xyz) · [**Install**](#install) · [**Docs**](docs/) · [**Benchmarks**](docs/benchmarks/juice-shop.md) · [**Agents**](https://github.com/0xSteph/pentest-ai-agents) · [**Discord**](https://discord.gg/6weeTAubJw)
+[**Website**](https://pentestai.xyz) · [**Install**](#install) · [**Why verification**](docs/why-verification.md) · [**Docs**](docs/) · [**Benchmarks**](docs/benchmarks/juice-shop.md) · [**Agents**](https://github.com/0xSteph/pentest-ai-agents) · [**Discord**](https://discord.gg/6weeTAubJw)
 
 </div>
 
 > ⚠️ **Offensive tooling, authorized testing only.** By installing you accept the [AUP](https://pentestai.xyz/aup) and [Terms](https://pentestai.xyz/terms). Full text in [Responsible use ↓](#responsible-use)
 
-Point ptai at a target. It runs recon, logs in, and ties findings into multi-step attack paths. Every finding comes with a working PoC. The report writes itself.
+ptai is an AI-driven pentest tool that re-runs every exploit to confirm it. It runs recon, logs in, and chains findings into multi-step attack paths, but it does not ask you to trust the results. The way TruffleHog confirms a leaked secret by logging in with it, ptai confirms a web finding by re-running the exploit: a finding stays a candidate until a machine oracle reproduces it N out of N, and only then does it earn a VERIFIED badge. Third-party scanner output (nuclei, nikto, zap) is held back until an oracle re-proves it. Scanner noise is what trains teams to ignore their tools, so the report carries only what ptai could prove, each VERIFIED finding with a portable proof capsule you can replay yourself.
 
-Runs on your laptop. No cloud, no telemetry.
+On a private 20-bug honeypot: 19/20 caught, 100% precision, zero false positives, four bug classes oracle-verified. Runs on your laptop. No cloud, no telemetry.
 
-> **What's new in 0.15.1 (2026-05-16).** Three new flags for pointing ptai at real production targets without breaking the engagement rules. `intensity=safe` skips probes that mutate server state (no mass-assignment, no stored XSS, no race conditions). `respect_rate_limits=true` honors HTTP 429 and `Retry-After` instead of hammering through them. `strict_scope=true` refuses any request whose host doesn't match the engagement target, and turns off redirect-following so a 302 to attacker.com can't pull the scan off-target. All three default off - existing 0.14.x behavior is unchanged unless you opt in. There's also a working `bearer` auth flow now: POST your creds, ptai pulls the JWT out of a JSON path you configure, then attaches it to every probe. JWT-style APIs (Juice Shop, crAPI, most modern bug-bounty stacks) actually work. Measured: **63.24% catch rate (43/68 in-scope challenges)** on OWASP Juice Shop v19.2.1 with `intensity=aggressive` + `strict_scope=true`. See [CHANGELOG](CHANGELOG.md#0151--2026-05-16).
->
-> Also in 0.14.0 (2026-05-13): 200+ wrapped security tools (wpscan, dalfox, hydra, hashcat, paramspider, ffuf, gobuster, sqlmap, ...) are reachable from Claude Code, Cursor, Codex, and any MCP client via `list_tools`, `run_tool`, `plan_tools`, `ensure_tools_installed`. **No Anthropic API key required.** Pick a tool, install missing ones in one batched prompt, run, get findings.
+## Prove it yourself
+
+<p align="center">
+  <img src="assets/ptai-demo.gif" alt="ptai demo: oracle-VERIFIED findings on a vulnerable app, then zero on the hardened twin" width="900">
+</p>
+
+```bash
+pip install ptai && ptai demo
+```
+
+`ptai demo` scans a bundled vulnerable app and reports `4 findings, 3 oracle-VERIFIED`, replays one live from a proof capsule (`replay 3/3`), then runs the same routes hardened and reports `0 findings`. The only thing that changed between the two runs is the fix, so the findings appear and disappear with the vulnerability, not because the tool went quiet. Two minutes, no API key, no target of your own. Re-prove any capsule yourself with `ptai replay`.
+
+> **Honest numbers.** The 19/20 at 100% precision figure is one run against a private 20-bug honeypot (39 findings total), not a field false-positive rate. The oracle gate buys precision without costing recall: it removes false positives, it does not raise catch rate. "Four oracle-verified classes" means four bug classes have a concrete oracle today; more land each release. OAST proves the HTTP-callback blind tier; DNS-only callbacks are a follow-on. The honeypot harness (`tests/honeypot/`) and a clean-app zero-FP gate (`tests/cleanapp/`) ship in the repo, so the claims are reproducible rather than screenshots.
+
+## What's new: the Trust Spine
+
+Every VERIFIED finding now comes from a named machine oracle, never an LLM assertion, and it is enforced in code: a verdict that cannot name its oracle is rejected. This release adds:
+
+- **Eight machine oracles** that re-run the exploit to confirm it: reflection, open-redirect, IDOR/BOLA, error-disclosure, MCP-exposure, SQLi (boolean and blind), and out-of-band (OAST) for blind SSRF/XXE over a self-hosted collaborator.
+- **Prove-or-kill gating**: third-party scanner output stays hidden until an oracle re-proves it.
+- **Portable proof capsules** with `ptai replay`, a live TUI that shows verdicts flip to VERIFIED on screen, and a CI gate (`--fail-on verified`) that breaks a build only on proven findings.
 
 ## See it run
 
 <p align="center">
-  <img src="assets/demo.gif" alt="ptai-via-Claude-Code-MCP scanning OWASP Juice Shop: 17 critical findings, 7 attack chains, 264 detection rules generated" width="900">
+  <img src="assets/demo.gif" alt="ptai scanning OWASP Juice Shop via Claude Code over MCP, streaming findings with working PoCs back into the session" width="900">
 </p>
 
 One prompt to Claude Code. The MCP server ran ptai's tools against the target, and Claude streamed findings back into the session. Subscription-driven, no API key.
 
-The scan returned **17 critical, 53 high, 107 total findings, 7 confirmed attack chains, and 264 generated detection rules** against a stock OWASP Juice Shop instance. JWT `alg:none` accepted on 8+ protected endpoints, SQLi auth bypass on `/rest/user/login`, UNION-based SQLi on `/rest/products/search`, path-filter bypass via NUL byte, XXE disclosing `/etc/passwd`, file upload polyglot, mass assignment, password reset bypass. Each one has a working PoC.
+Pointed at a stock OWASP Juice Shop through Claude Code over MCP, ptai found and PoC'd the headline bugs: JWT `alg:none` accepted on 8+ protected endpoints, SQLi auth bypass on `/rest/user/login`, UNION-based SQLi on `/rest/products/search`, path-filter bypass via NUL byte, XXE disclosing `/etc/passwd`, file-upload polyglot, mass assignment, password reset bypass. Each one has a working PoC. Juice Shop is a heavily-studied target, so read the raw finding volume as breadth; the precision story, only what an oracle could prove, is the demo and honeypot above.
 
 > Recording is the actual output of `claude -p` against a local OWASP Juice Shop with `pentest-ai` registered as an MCP server. Cast file in [`assets/realdemo.cast`](assets/realdemo.cast); the time-paced re-render used for the GIF is in [`assets/realdemo-paced.cast`](assets/realdemo-paced.cast). Findings are real; inter-line timing was reconstructed for watchability since `claude -p` buffers and dumps in non-interactive mode. A deterministic synthesized fallback ([`assets/demo.tape`](assets/demo.tape) + [`assets/demo.sh`](assets/demo.sh)) is kept for reproducible re-renders.
 

action.yml

@@ -0,0 +1,43 @@
+name: "ptai scan"
+description: "Run ptai's deterministic web sweep and fail the build only on oracle-VERIFIED findings."
+author: "0xSteph"
+branding:
+  icon: "shield"
+  color: "purple"
+
+inputs:
+  target:
+    description: "Target URL to scan (must be in scope / authorized)."
+    required: true
+  fail-on:
+    description: "Build gate: 'verified' (fail only on an oracle-VERIFIED finding), 'high' (fail on any critical/high finding), or 'never'."
+    required: false
+    default: "verified"
+  intensity:
+    description: "stealth | safe | normal | aggressive."
+    required: false
+    default: "normal"
+  python-version:
+    description: "Python version to run ptai on."
+    required: false
+    default: "3.12"
+
+runs:
+  using: "composite"
+  steps:
+    - uses: actions/setup-python@v5
+      with:
+        python-version: ${{ inputs.python-version }}
+    - name: Install ptai
+      shell: bash
+      run: pipx install ptai || pip install --user ptai
+    - name: Scan and gate
+      shell: bash
+      # PENTEST_AI_AUP_ACCEPTED: you assert the target is authorized for testing.
+      env:
+        PENTEST_AI_AUP_ACCEPTED: "1"
+      run: >
+        ptai start "${{ inputs.target }}"
+        --scope web --no-llm --ci --no-sync
+        --intensity "${{ inputs.intensity }}"
+        --fail-on "${{ inputs.fail-on }}"