Skip to content

Clarify inline verifier invocation envelopes - #929

Closed
yuelgrace1810-ops wants to merge 8 commits into
mainfrom
agent/clarify-inline-verifier-envelope
Closed

Clarify inline verifier invocation envelopes#929
yuelgrace1810-ops wants to merge 8 commits into
mainfrom
agent/clarify-inline-verifier-envelope

Conversation

@yuelgrace1810-ops

Copy link
Copy Markdown
Collaborator

What changed

  • Advertise a concrete math.run invocation envelope for inline replay verifiers in both discovery results and full contract inspection.
  • State explicitly that input and candidate are sibling fields inside payload.
  • Correct the graded Jacobian-syzygy verifier description, which previously claimed it consumed a stored producer result even though its contract accepts an inline producer input/result pair.
  • Cover the structured and model-visible text projections with MCP boundary tests.

Why

A held-out weak-model evaluation using the exact degree-8 binary partition-polynomial log-concavity counterexample found the correct Jacobian syzygy immediately, but then spent 408 seconds and failed two verifier calls. The model first nested candidate inside input, then placed it beside payload. Both invalid shapes are plausible readings of a contract that had no invocation example and contradictory stored-versus-inline wording.

This change exposes transport shape without prescribing mathematical strategy, making producer-to-verifier replay easier and reducing avoidable validation retries.

Validation

  • make test-unit — 869 passed
  • make test-component — 759 passed, 3 skipped
  • make test-domain — 185 passed
  • make test-mcp — 46 passed
  • make test-e2e — 8 passed, 1 skipped
  • make check-static — passed, including lint, format, dependency, type, architecture, and build checks
  • Focused Jacobian syzygy producer/verifier test — passed

make npm-test could not run because npm is not installed on this host.

Copy link
Copy Markdown
Collaborator Author

Additional held-out reproduction on current main (2026-08-09): a gpt-5.4-mini/low run audited the exact Dimca–Pokora nine-line Generalized Terao counterexample (arXiv:2607.01985). It successfully discovered, executed, and independently verified both geometry.projective_line_arrangement.flats artifacts. For polynomial.jacobian_syzygy.minimum_degree, it computed the correct mdr(f)=4 and mdr(g)=5, but made four invalid verifier attempts: bare result_uri twice, then {input, candidate:{result_uri}}, then {input, candidate:<materializer wrapper>}. It stopped at COMPUTED assurance despite the verifier contract being inspected first. Valid grounded runtime was 185.84s / 20 MCP calls / 413,125 input tokens, versus control 112.40s / 171,792 input tokens. This is the same invocation-envelope ambiguity addressed by this PR, not a separate defect.

Copy link
Copy Markdown
Collaborator Author

Post-#924 recovery evidence (2026-08-09): the same weak-model configuration (gpt-5.4-mini, low) successfully replayed the frozen zero-partial Gao coordinate task on current main. It found the producer/verifier pair, produced the exact degree-0 certificate for H=-x^3+x^2z, and passed the producer's clean inline input/result to the verifier on the first attempt; the verifier returned VERIFIED. End-to-end: 61.50s, 5 MCP calls, 137,364 input tokens. This does not isolate the effect of this PR because the branch was not applied, but it confirms that a clearly understood sibling-field envelope is sufficient and provides a clean comparison point for the earlier invalid shapes.

Copy link
Copy Markdown
Collaborator Author

Branch validation against the original frozen gpt-5.4-mini/low partition-polynomial case exposed an important dependency boundary. The #929-only branch discovered the producer/verifier and produced the correct degree-0 result, but current main's producer still advertised verification_input_field: "result_uri"; the model then made an invalid verifier call and the 600 s arm timed out. This does not contradict the envelope fix here—the stale producer hint is the separate defect already fixed by draft PR #917. I refreshed #917 onto current main and validated its focused syzygy tests (2 passed) plus the full MCP lane (47 passed). A combined post-fix model replay should wait until #917 is present alongside this branch.

@morluto
morluto marked this pull request as ready for review August 9, 2026 17:38

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b42d08c468

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/jacobian/adapters/mcp/projections.py
@morluto
morluto force-pushed the agent/clarify-inline-verifier-envelope branch from b42d08c to c4ae444 Compare August 9, 2026 17:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4ae4443db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/jacobian/adapters/mcp/projections.py Outdated
Comment thread src/jacobian/adapters/mcp/tools.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ce090e268

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/jacobian/adapters/mcp/projections.py Outdated

morluto commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Second-pass architecture review: rebase + likely merge. This survives the post-#956 architecture well: it exposes the concrete transport shape of an already-declared producer/verifier relationship without prescribing verification timing or research strategy. The earlier reasoning-ID concern is obsolete because #956 removed the production reasoning-mode schema. The later review hardening around typed invocation-protocol models, explicit producer relationships, and response.output.result targeting is the right direction.

This was referenced Aug 10, 2026
# Conflicts:
#	src/jacobian/domains/polynomial/checkers.py
#	tests/boundary/mcp/test_mcp_adapter.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e8f9fbd8e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/jacobian/adapters/mcp/projections.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a70e7c800

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/boundary/mcp/test_mcp_discovery_projection.py Outdated

@morluto morluto left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-cutover architecture audit

Disposition: revise substantially; retain the invocation-example goal.

The observed payload-shape failure is valid, but this implementation derives a new MCP invocation_protocol projection from a deleted producer relationship and schema heuristics. That recreates transport-specific metadata and another source of truth.

On current main, put a concrete {capability_id, payload: {input, candidate}} invocation example directly on each applicable checker OperationSpec/descriptor. Exact inspection already returns the full descriptor, so no synthetic projection or relationship test is needed. Correct the syzygy description at the checker declaration, add one generic invariant that advertised examples validate against math.run, and delete all relationship/projection code from this branch.

morluto commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Architecture review after #1256/#1265: superseded as an implementation; keep the usability problem and solve it through current descriptor examples.

The observed failure is valid: inline replay checkers with sibling input / candidate fields are easy for agents to wrap incorrectly. But this PR derives a special invocation_protocol from descriptor.related_capabilities / CapabilityCatalogRelationshipKind. Those relationship models were deliberately removed in #1256 along with workflow-like catalog navigation.

Current exact inspection already has authoritative request schemas and bounded, operator-authored invocation_examples. That is the simpler owner for transport-shape guidance: give affected checker declarations one schema-valid example showing {capability_id, payload: {input, candidate}}, and let exact inspection expose it normally. No relationship graph or verifier-specific MCP projection is needed.

Recommendation: do not rebase this patch. Replace it with a much smaller current-main change adding validated invocation examples to the affected checker descriptors (and an MCP inspection regression), then close #929 as superseded.

morluto commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Superseded by typed ports, operation examples, and current lineage cleanup

Closing this pre-cutover implementation rather than rebasing its generic projection heuristic.

The reproduced failure was real, but the surviving solution is now split across current ownership boundaries:

This branch inferred an invocation_protocol from catalog relationships and request-field names. That recreates transport policy in the discovery projector and depends on relationship machinery deliberately removed by #1256/#1265. New verifier examples or ports should be declared by the operation/checker owner, not synthesized by MCP projection.

Disposition: root cause retained; implementation superseded.

@morluto morluto closed this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants