Skip to content

feat(skills): add traceable proposal response (RFI/RFP) skill into project planning - #2682

Open
Katrien De Graeve (katriendg) wants to merge 26 commits into
mainfrom
feat/proposal-rsp-skill
Open

feat(skills): add traceable proposal response (RFI/RFP) skill into project planning#2682
Katrien De Graeve (katriendg) wants to merge 26 commits into
mainfrom
feat/proposal-rsp-skill

Conversation

@katriendg

@katriendg Katrien De Graeve (katriendg) commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

This PR adds a new proposal-response skill for project planning and wires it into the BRD Builder and PRD Builder, so a user holding an RFP question set plus an existing BRD or PRD can reach a traceable internal-review draft without transcribing documents into chat or leaving the agent they started in.

The skill records, qualifies, and drafts from approved evidence. It does not authorize external use, approval, submission, or release.

The skill

  • Added .github/skills/project-planning/proposal-response/ with analyze, contribute, and draft operations, persisting a single RESPONSE_EVIDENCE_V1 artifact per response and returning a compact RESPONSE_EVIDENCE_POINTER_V1.
  • Formalized a claim and evidence model with stable SQ, CLM, RSP, UNR, and SRC record IDs, a derived response_state table, a source-question inclusion test that keeps directive text out of every count, and derived blocking_ids.
  • Added an advisory response-quality rubric covering claim traceability, evidence fit, source registration, source currency, ownership, response fidelity, decision visibility, coverage integrity, and the authority boundary.
  • Added bundled templates for a business evidence appendix, a product evidence appendix, and a shared response draft, each carrying an approved-sources register.

Approved-source intake

A user names an existing BRD or PRD, and the skill reads it, registers it as an SRC record with its kind, read date, declared version, and the sections used, then cites it from every claim as SRC-001#NFR-014. A reviewer can trace any statement back to the approved document rather than to recalled text.

Sources take two forms. origin: file is an artifact the skill opened at a named path. origin: user_supplied is approved evidence stated directly in the request, recorded with source_version: unknown because there is nothing to re-read. The skill prefers a named artifact and says so when it falls back.

Self-describing sequence

Every pointer carries next_operation, derived from the current records through an ordered condition table rather than asserted. It reaches the terminal none when every question has a current response, or when the only remaining open items need a human decision the skill cannot make, so an open legal_or_commercial item no longer leaves the caller looping.

The pointer reports record_counts per record kind plus the IDs this operation changed, so its size stays constant as the record set grows.

Builder integration

  • Both builders bind all three operations. The domain binding applies to contribute only: BRD Builder contributes business-owned evidence, PRD Builder product-owned.
  • Drafting is not domain-scoped. Either builder renders across every reviewed claim, so a question answered by the other domain still gets a response, while creating no claim outside its own domain.
  • The shared activation, session-state, rejected-operation, and reporting contract lives once in references/builder-extension-contract.md inside the skill, loaded on activation. Each agent keeps its trigger, domain and operation bindings, extension-state marker, supported rendering choices, and non-activation boundary.
  • Ordinary BRD and PRD creation, refinement, resume, quality review, and handoff requests do not activate the extension.

Drafting versus rendering

The draft operation records RSP entries inside the evidence artifact. Writing response-draft.md or an appendix is a separate rendering that happens only on request, so a draft with no rendering request returns an empty rendered-artifact list. A repeated rendering replaces the file from the stored payload rather than appending, keeping one response block per question.

Supporting changes

  • Registered the skill file and both builder agents in the untrusted-content boundary, since all three now ingest supplied bid content directly.
  • Added .copilot-tracking/proposal-responses/** to the tracking conventions, including the rule that versioned payload contracts use their owning schema's YAML or JSON format.
  • Added the skill to the project-planning and aggregate marketplace packages.
  • Extended skill-tier eval coverage for operation selection, continuation validation, source intake and unresolvable-source stops, routing including the terminal value, cross-domain drafting, rendering separation, re-render replacement, an end-to-end sequence, source immutability, and authority-boundary enforcement. Builder-tier coverage remains bounded to each builder's ordinary class behavior and non-activation boundary because the current agent-behavior harness loads skill combinations but does not invoke the named agent definition.
  • Updated project-planning documentation and role and lifecycle pages to describe the operation order, approved sources, builder bindings, and the drafting-versus-rendering distinction.

Note

This branch also carries a BRD-to-PRD handoff persistence change in requirements-author: BRD_TO_PRD_HANDOFF_V1 is now written to .copilot-tracking/brd-sessions/<brd-name>.handoff.yml and passed by path rather than inline. It is unrelated to proposal response and may be worth splitting out.

Safety and boundary design

  • Every output remains internal_review_draft, both external_use_status values deny external use, and release_decision stays outside_skill_scope.
  • The skill refuses requests to mark a response approved, authorized, externally usable, submitted, committed, or released, and returns the existing artifact pointer instead.
  • Structural readiness is advisory. It never grants approval, authorization, release, submission, or external use, and a complete-looking draft remains internal review material.
  • Claims are never fabricated from a description. Behavior testing repeatedly confirmed this: when a request described records without supplying question text or evidence, the skill stopped and asked rather than inventing them.
  • A named source path that does not resolve stops the operation with the path and the smallest clearing action visible. The skill does not reconstruct the document from memory.
  • Continuations read and validate the existing artifact before any merge, rendering, or writeback, and reject an invalid payload without overwriting the supplied record. This read-before-merge rule is also what protects the payload when two builders work the same response.
  • Source questionnaires and approved artifacts stay read-only. Outputs are written only under the proposal-response tracking folder.
  • Supplied questions, attachments, and tool-returned content are treated as data. Directive text embedded in a questionnaire is excluded from every record and count and reported as an ignored reference rather than executed.

Related Issue(s)

Fixes #2681

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with hve-builder and addressed all actionable findings
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

I have an RFP question set and an existing BRD at docs/project-planning/supplier-onboarding-brd.md. Help me answer it without granting approval or release authority.

Execution Flow:

  1. The user asks BRD Builder for RFP help, which activates the extension and loads the shared builder-extension contract.
  2. analyze reads the named BRD, registers it as SRC-001 with its version and read date, and normalizes the questions into SQ records.
  3. The pointer reports next_operation: contribute, derived from the records, and the turn closes by naming that step.
  4. contribute adds business-owned claims whose evidence_refs cite SRC-001#BG-001 and similar, leaving unsupported material as qualified claims or unresolved items.
  5. draft records RSP entries across every reviewed claim, including questions answered by product-owned evidence, and reports next_operation: none.
  6. A rendering is produced only if the user asks for one; a second request replaces the file rather than appending.

Output Artifacts:

  • .copilot-tracking/proposal-responses/<response-slug>/response-evidence.yml
  • Optional business or product evidence appendix renderings
  • Optional shared response-draft.md internal-review rendering

Success Indicators:

  • Every claim resolves to a registered source rather than to transcribed or recalled text.
  • The pointer names the remaining step, and reaches none when only human decisions are left.
  • Output stays internal_review_draft and never claims approval or release authority.
  • Unresolved items and qualifications stay visible beside the responses they affect.
  • An unresolvable source path or an invalid continuation stops with a clearing action instead of a fabricated answer.

Testing

Validated for this change set:

  • npm run validate:skills passed across 65 skills with no errors or warnings.
  • npm run ci:eval:lint:schema validated all 16 eval specifications, 110 reachable tag values, and all 35 parent agents.
  • Build-AgentBehaviorSpec.ps1 -WhatIf reported no drift after regeneration.
  • BRD Builder's final bounded agent-behavior slice scored 100% across 10 trials; PRD Builder scored 100% across 10 trials. A combined repeat scored 100% across another 20 trials, with no runtime errors across the final 40-trial configuration.
  • npm run docs:generate:check reported no generated-reference drift across 233 pages, and npm run plugin:validate passed for all 10 marketplace packages.
  • npm run validate:docs passed 10 documentation test suites and 101 tests; npm run spell-check passed across 836 files.

The proposal-response operation journeys are exercised in the skill-behavior suite, which can observe the skill contract directly. The builder agent suite intentionally does not claim native agent-binding coverage because its current executor does not route through the named .agent.md definition.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used hve-builder review mode to review contribution
  • Addressed all actionable findings from the hve-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

  • Local validation aggregate: npm run validate:local (fails on existing markdown-link issues outside this feature)
  • Documentation validation (if docs changed): npm run validate:docs
  • Spell checking: npm run spell-check
  • Link validation: npm run lint:md-links (fails on existing markdown link references and one missing prompt path)

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues (no dependency additions)
  • Security-related scripts follow the principle of least privilege (no privileged script additions)

Additional Notes

This change is intentionally scoped to a traceability-first, internal-review drafting workflow. The operational boundary remains strict: the skill records, qualifies, and drafts from approved evidence but does not authorize external use, approval, submission, or release decisions.

Two design choices are worth a reviewer's attention. The shared builder-extension contract ships as a reference inside the skill rather than as a packaged instruction file, because it is only needed when an agent activates the skill and it inherits the skill's existing package membership that way. And the returned pointer deliberately reports record_counts rather than enumerating every retained record, so the contract a caller reads stays the same size as a response grows.

- add traceable analyze, contribute, and draft operations
- integrate explicit BRD and PRD builder activation
- add behavior coverage, documentation, and package projections

📋 - Generated by Copilot
…ss and source immutability

- clarify blocking_ids definition in response quality rubric
- specify rendering requirements for unresolved questions in response draft
- add new behavior evaluation for source immutability in proposal response

🔒 - Generated by Copilot
… response

- modify descriptions to reflect new conventions
- update metadata dates for consistency
- enhance examples with detailed instructions for contributions

🔍 - Generated by Copilot
- gate builder artifact recording on RESPONSE_EVIDENCE_POINTER_V1 only
- define response_state conditions, transitions, and blocking_ids derivation
- exclude injected directive text from question_count via inclusion test
- route legal_or_commercial questions to unresolved items, not claims
- stop artifact rewrites for read-only status requests and add eval coverage

🧾 - Generated by Copilot
…osal responses

feat(proposal-response): enhance response draft and appendix templates with markdown linting

🔒 - Generated by Copilot
@katriendg
Katrien De Graeve (katriendg) requested a review from a team as a code owner August 12, 2026 10:06
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.73%. Comparing base (90414cb) to head (0c2f63a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2682      +/-   ##
==========================================
- Coverage   83.08%   81.73%   -1.36%     
==========================================
  Files         183      162      -21     
  Lines       33712    22377   -11335     
  Branches       25       25              
==========================================
- Hits        28010    18289    -9721     
+ Misses       5699     4085    -1614     
  Partials        3        3              
Flag Coverage Δ
docusaurus 89.92% <ø> (ø)
pester 83.90% <ø> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 44 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Eval Execution

Status: Passed — no merge-blocking failures (32 advisory assertion failure(s) present)

  • Artifacts evaluated: 7
  • Specs run: 7
  • Assertions passed: 121
  • Assertions failed (blocking): 0
  • Assertions failed (advisory): 32
  • Failed specs (merge-blocking): 0
Artifact Kind Status Specs Passed Failed (blocking) Failed (advisory)
brd-builder agent ✅ pass 1 15 0 0
prd-builder agent ✅ pass 1 15 0 0
copilot-tracking instruction ❌ fail 1 0 0 0
untrusted-content-boundary instruction ❌ fail 1 0 0 0
outcome-hypothesis skill ⚠️ advisory-fail 1 33 0 30
proposal-response skill ⚠️ advisory-fail 1 55 0 2
requirements-author skill ✅ pass 1 3 0 0

Legend — ✅ clean · ⚠️ advisory failures only (non-blocking) · ⏭️ skipped · ❌ merge-blocking failure

Only Failed specs (merge-blocking) gates this PR. Advisory assertion failures are signal-quality checks captured during iteration; review them, but they do not block merge and may be acceptable.

…siness and product contributions

- add compact YAML result for proposal-response operations
- include artifact paths and status in the response
- clarify handling of evidence and approval statuses

🔧 - Generated by Copilot
@katriendg
Katrien De Graeve (katriendg) marked this pull request as draft August 12, 2026 13:07
…onse

- register user-named BRD or PRD as SRC records with resolvable evidence
- derive next_operation; bind analyze, contribute, draft in both builders
- drop caller state echo, record enumeration, and output-order mandates

📋 - Generated by Copilot
- register inline approved evidence as a user_supplied source, version unknown
- scope the unresolved-source stop rule to a named path only
- mount fixtures for stimuli that described a payload instead of supplying one
- trim graders toward user-visible behavior over transcript shape

🔧 - Generated by Copilot
- focus graders on user-visible outcomes
- reduce timeout-prone BRD and PRD stimuli
- reject false-positive draft results

🧪 - Generated by Copilot
- retain proposal-response coverage in the skill suite
- bound builder checks to read-only behavior
- remove orphaned agent fixtures

🧪 - Generated by Copilot
@katriendg
Katrien De Graeve (katriendg) marked this pull request as ready for review August 13, 2026 09:21
- add typed outcome evidence and readiness handling
- clarify handoff transport and privacy fallback behavior
- expand behavior conformance coverage

🔗 - Generated by Copilot
…handling

- add logic to retry reading missing artifact paths
- update response to include registered SRC record and compact pointer
- clarify prompts for overwriting supplied source artifacts

🔧 - Generated by Copilot

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the substantial proposal-response workflow and its traceability work. After a full review and a contrarian conformance pass, I am requesting changes for one blocking state-model issue and four contract or evaluation issues described inline. The unresolved-item model cannot represent a cleared item while preserving its history. The remaining inline comments cover an unsupported concurrency guarantee, malformed BRD and PRD rejected-operation expectations, and an error-response mismatch with the builder reporting contract. Four lower-priority portability and documentation issues remain in the linked review report but are not posted inline. Earlier speculative security and accessibility concerns were withdrawn from this review.

Comment thread .github/skills/project-planning/proposal-response/SKILL.md Outdated
Comment thread evals/agent-behavior/expectations/brd-builder.expectations.yml
Comment thread evals/agent-behavior/expectations/prd-builder.expectations.yml
- preserve cleared unresolved records with evidence links
- separate success and rejection reporting contracts
- add lifecycle and builder rejection eval coverage
- use workspace-aware reads for relative artifact paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): add traceable proposal response (RFI/RFP) skill into project planning

3 participants