Complete scalable MCP contracts and PR health coverage

docs/architecture.md

@@ -77,10 +77,20 @@ The corpus separates source history from convenient current state:
   revision it represents.
 
 Authored pull requests use the ordinary repository and thread projections.
-REST `pr_details` and `pr_reviews` facets add mergeability, revision, and review
-facts for the portfolio view. Checks, unresolved review threads, detailed merge
-state, and merge-queue state are not currently acquired and therefore remain
-explicitly unavailable rather than inferred.
+REST `pr_details` and `pr_reviews` facets are combined with typed GraphQL
+facets for checks, unresolved review threads, detailed merge state, merge queue,
+closing issues, and changed files. Each facet has independent coverage; an
+incomplete refresh preserves the previous complete child snapshot but marks
+the newer coverage incomplete. Offline portfolio reads therefore return
+`unknown` instead of treating missing checks as passing or missing overlap
+signals as no overlap.
+
+Portfolio relationships and derived resolution records are local product
+contracts. Their normalized snapshots carry rule versions and exact source
+observation references. Explicit timeline events may produce a resolution;
+closing-issue relationships remain relationship evidence until completion is
+independently observed. Lexical similarity alone never becomes a root-cause
+claim. Corpus portfolio and resolution reads perform no network access.
 
 Repository and thread projections use this ordering:
 
@@ -128,6 +138,10 @@ Terminal states do not transition again. Cancellation is first persisted, then
 delivered to an in-process worker directly or observed by its polling loop from
 another process. Reconciliation uses an immediate SQLite transaction so a
 heartbeat cannot interleave between the liveness read and stale-owner update.
+MCP job reads expose structured phase, completed-item, total-item, percentage,
+and retry-delay fields. Batch reads and cancellation preserve input order and
+isolate per-item failures; free-form durable event text is not an MCP progress
+contract.
 
 ### Bounded batch operations
 

docs/mcp-scalable-workflows.md

@@ -36,19 +36,27 @@ github.get_authenticated_identity
 -> github.sync_authored_pull_requests -> jobs.get
 -> github.sync_pull_request_status -> jobs.get
 -> corpus.list_pull_request_portfolio
+-> corpus.find_portfolio_overlaps
 ```
 
-The current status adapter stores REST pull-request details and reviews. The
-portfolio can classify merged, closed-unmerged, conflicted, changes-requested,
-approved, stale, awaiting-review, and unknown states from those facts.
-
-The following facts are not currently fetched and remain explicit in coverage
-and reason fields:
-
-- check rollups and failing checks;
-- unresolved review conversations;
-- detailed merge state and merge queue position;
-- closing issue links and cross-portfolio overlap.
+The status adapter stores REST pull-request details and reviews plus typed,
+independently covered GraphQL snapshots for checks, unresolved review threads,
+detailed merge state, merge queue, closing issues, and changed files. The
+offline portfolio derives deterministic attention states only from complete
+facets. A null or still-computing mergeability value remains unknown.
+
+`corpus.find_portfolio_overlaps` compares up to 50 stored candidates with
+authored pull requests using complete normalized changed-path, linked-issue,
+and stored opportunity-similarity evidence. It returns `unknown` unless every
+required facet is complete; it never performs network access. Use
+`workflow.link_pull_request` to record an explicit local PR association with an
+opportunity or workspace. That local write does not mutate GitHub.
+
+Issue timeline hydration is an explicit, opt-in `issue_timeline` facet. Complete
+timeline observations may create versioned resolution records with exact source
+observation references. Closing-issue observations remain relationship evidence
+until completion is independently observed. Similar prose is not resolution
+evidence.
 
 `workspace.check_merge_conflicts` is different from GitHub mergeability. It runs
 a non-mutating Git comparison between already-fetched object IDs in a managed
@@ -69,6 +77,11 @@ jobs together with vectorized `jobs.get`, then retry only retryable items. Never
 interpret absent coverage as a zero, a passing check, or a lack of competing
 work.
 
+`corpus.get_coverage` accepts up to 100 ordered repository or exact-thread
+targets. `jobs.cancel` accepts up to 100 IDs and returns isolated item outcomes;
+repeating cancellation is safe. `jobs.get` exposes structured phase and item
+counts rather than requiring clients to parse event prose.
+
 The MCP catalog does not advertise scalar compatibility aliases. Use one-item
 arrays with `corpus.get_repositories`, `corpus.get_threads`,
 `github.sync_threads`, `github.hydrate_threads`, and `jobs.get` when only one
@@ -80,6 +93,7 @@ not an MCP discovery primitive.
 | Tool family | Network | Corpus/local write | Process |
 | --- | ---: | ---: | ---: |
 | `corpus.get_*`, rank, precedents, portfolio | no | no | no |
+| `workflow.link_pull_request` | no | yes | no |
 | `github.search_*`, sync, hydrate | yes | yes | no |
 | `research.query_deepwiki` | yes | no | no |
 | `code.index_repositories` | remote-dependent | yes | Git only |

docs/scalable-research-and-portfolio-plan.md

@@ -2,22 +2,22 @@
 
 Status: partially implemented
 
-The bounded metadata, thread, Radar, precedent, authored-PR, REST status,
-DeepWiki, code-indexing, local conflict, and vectorized job-read tools are
-implemented. This document also retains target contracts that are not yet
-implemented; the table below is authoritative for current coverage.
+The bounded metadata, thread, Radar, precedent, authored-PR, PR-health,
+portfolio-relationship, DeepWiki, code-indexing, local conflict, and vectorized
+job tools are implemented. This document also retains longer-term target
+contracts; the table below is authoritative for current coverage.
 
 | Area | Status | Current coverage |
 | --- | --- | --- |
 | Batch repository and thread reads | Implemented | Ordered item results, typed nullable metadata, compact/full threads |
 | Metadata, thread, and selected-facet synchronization | Implemented | Durable bounded jobs with server-side concurrency |
 | Cross-repository Radar and historical precedents | Implemented | Offline ranking and direct similarity over stored resolved threads |
 | DeepWiki | Implemented | One bounded non-persisting external-read primitive |
-| Authored PR discovery and portfolio | Partial | Identity, authored search, REST details/reviews, deterministic attention |
-| PR health | Deferred | Checks, unresolved review threads, detailed merge state, merge queue, closing issues |
-| Portfolio relationships | Deferred | Cross-PR overlap and explicit opportunity/workspace links |
-| Remaining vectorization | Deferred | Batch coverage reads and batch job cancellation |
-| Rich derived resolutions | Deferred | Timeline/file/closing-PR facets and persisted resolution projections |
+| Authored PR discovery and portfolio | Implemented | Identity, authored search, REST details/reviews, typed health facets, deterministic attention |
+| PR health | Implemented | Checks, unresolved review threads, detailed merge state, merge queue, closing issues, changed files |
+| Portfolio relationships | Implemented | Offline normalized overlap and explicit opportunity/workspace links |
+| Remaining vectorization | Implemented | Batch coverage reads, job reads, and idempotent batch cancellation |
+| Rich derived resolutions | Implemented | Opt-in timeline-derived projections plus changed-file/closing-issue relationship facets |
 
 See [Scalable MCP workflows](mcp-scalable-workflows.md) for the current tool
 sequence, recovery rules, test boundary, and limitations.
@@ -295,8 +295,8 @@ filters. Repository mode does not accept exact thread references.
 
 An empty facet list must be rejected. "Everything" is not a safe default.
 
-`github.sync_pull_request_status` should prefer one bounded GraphQL query per
-batch and project at least:
+`github.sync_pull_request_status` uses bounded typed GraphQL reads per pull
+request and projects:
 
 - state and draft state;
 - author and repository identity;
@@ -309,9 +309,9 @@ batch and project at least:
 - closing issue references;
 - updated, closed, and merged times.
 
-REST adapters remain useful for paginated child facets and fallback behavior.
-GitHub `mergeable: null` is returned as a retryable item with a suggested delay,
-not as a terminal error or a persisted `false`.
+REST adapters remain responsible for details, reviews, and issue timelines.
+GraphQL collection pagination is bounded by the job input. GitHub
+`mergeable: null` remains unknown rather than becoming persisted `false`.
 
 ### 4.3 DeepWiki adapter
 
@@ -620,16 +620,11 @@ against:
 - linked issues from authored pull requests;
 - explicit cross-references;
 - changed-file paths when that facet is present;
-- local hypothesis and opportunity text;
-- deterministic thread similarity;
-- currently competing upstream pull requests.
+- stored opportunity-similarity signals.
 
-The output distinguishes:
-
-- Git merge conflict;
-- competing upstream implementation;
-- overlap with the user's own portfolio;
-- weak textual similarity.
+The output distinguishes exact observed overlap, complete no-overlap, and
+unknown coverage. Local merge conflicts and competing upstream work remain
+separate primitives rather than being inferred by this tool.
 
 Opportunity ranking should exclude or clearly mark candidates already covered
 by the user's work.

internal/app/hydration.go

@@ -19,6 +19,13 @@ const (
 	FacetPRDetails        = "pr_details"
 	FacetPRReviews        = "pr_reviews"
 	FacetPRReviewComments = "pr_review_comments"
+	FacetPRChecks         = "pr_checks"
+	FacetPRReviewThreads  = "pr_review_threads"
+	FacetPRMergeState     = "pr_merge_state"
+	FacetPRMergeQueue     = "pr_merge_queue"
+	FacetPRClosingIssues  = "pr_closing_issues"
+	FacetPRFiles          = "pr_files"
+	FacetIssueTimeline    = "issue_timeline"
 )
 
 var issueFacets = []string{FacetIssueComments}
@@ -166,6 +173,8 @@ func (s *Service) HydrateThread(ctx context.Context, repo cli.RepoRef, number in
 			facetResult, err = f.hydratePullRequestReviews()
 		case FacetPRReviewComments:
 			facetResult, err = f.hydratePullRequestReviewComments()
+		case FacetIssueTimeline:
+			facetResult, err = f.hydrateIssueTimeline()
 		default:
 			hydrateErr = fmt.Errorf("unknown facet %q", facet)
 			return nil, hydrateErr
@@ -208,6 +217,9 @@ func selectFacets(kind string, requested []string) ([]string, error) {
 	if len(requested) == 0 {
 		return allowed, nil
 	}
+	// Timeline history is intentionally opt-in because it can be much larger
+	// than the default hydration set.
+	allowed = append(append([]string(nil), allowed...), FacetIssueTimeline)
 
 	allowedSet := make(map[string]struct{}, len(allowed))
 	for _, f := range allowed {
@@ -229,6 +241,119 @@ func selectFacets(kind string, requested []string) ([]string, error) {
 	return out, nil
 }
 
+func (f *facetRunner) hydrateIssueTimeline() (HydratedFacet, error) {
+	reader, ok := f.reader.(github.IssueTimelineReader)
+	if !ok {
+		return HydratedFacet{}, errors.New("GitHub reader does not support issue timelines")
+	}
+	opts := github.PageOptions{Page: 1, PerPage: 100}
+	var total, pages int
+	var complete bool
+	var pageObservations []corpus.FacetObservationInput
+	sourceUpdatedAt := f.thread.SourceUpdatedAt
+	var events []github.IssueTimelineEvent
+	for pages < f.maxPages {
+		if err := f.ctx.Err(); err != nil {
+			return HydratedFacet{}, err
+		}
+		res, err := reader.ListIssueTimeline(f.ctx, f.ref.Owner, f.ref.Repo, f.thread.Number, opts)
+		if err != nil {
+			return HydratedFacet{}, err
+		}
+		pages++
+		pageUpdatedAt := sourceUpdatedAt
+		for _, event := range res.Items {
+			if event.CreatedAt.After(pageUpdatedAt) {
+				pageUpdatedAt = event.CreatedAt
+			}
+		}
+		payload, err := json.Marshal(res.Items)
+		if err != nil {
+			return HydratedFacet{}, fmt.Errorf("marshal issue timeline: %w", err)
+		}
+		pageObservations = append(pageObservations, corpus.FacetObservationInput{SourceUpdatedAt: pageUpdatedAt, Payload: string(payload)})
+		events = append(events, res.Items...)
+		total += len(res.Items)
+		if pageUpdatedAt.After(sourceUpdatedAt) {
+			sourceUpdatedAt = pageUpdatedAt
+		}
+		if !res.Page.HasNext {
+			complete = true
+			break
+		}
+		opts.Page = res.Page.NextPage
+	}
+	if !complete {
+		if err := f.c.AdvanceFacet(f.ctx, f.repoID, &f.threadID, FacetIssueTimeline, sourceUpdatedAt, false, f.runID); err != nil {
+			return HydratedFacet{}, err
+		}
+		return HydratedFacet{Facet: FacetIssueTimeline, Count: total, Pages: pages, Complete: false}, nil
+	}
+	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetIssueTimeline, sourceUpdatedAt, pageObservations, true, f.runID); err != nil {
+		return HydratedFacet{}, err
+	}
+	coverage, err := f.c.GetCoverage(f.ctx, f.repoID, &f.threadID, FacetIssueTimeline)
+	if err != nil {
+		return HydratedFacet{}, err
+	}
+	if coverage == nil || !coverage.Complete || !coverage.SourceUpdatedAt.Equal(sourceUpdatedAt.Truncate(time.Second)) {
+		// A newer stored snapshot won the stale-write comparison. Do not attach
+		// this older derivation to that snapshot's observation identities.
+		return HydratedFacet{Facet: FacetIssueTimeline, Count: total, Pages: pages, Complete: true}, nil
+	}
+	if err := f.persistTimelineResolution(events, sourceUpdatedAt); err != nil {
+		return HydratedFacet{}, err
+	}
+	return HydratedFacet{Facet: FacetIssueTimeline, Count: total, Pages: pages, Complete: true}, nil
+}
+
+func (f *facetRunner) persistTimelineResolution(events []github.IssueTimelineEvent, sourceUpdatedAt time.Time) error {
+	kind, summary := "", ""
+	selectedCommit := ""
+	if f.thread.StateReason == "not_planned" {
+		kind, summary = "not_planned", "GitHub records this issue as closed without planned work."
+	}
+	for _, event := range events {
+		if event.Event == "closed" && event.CommitID != "" {
+			kind, summary = "fixed_by_commit", "GitHub records an explicit closing commit: "+event.CommitID
+			selectedCommit = event.CommitID
+		}
+	}
+	if kind == "" {
+		return nil
+	}
+	var refs []corpus.ObservationRef
+	if selectedCommit == "" {
+		observation, err := f.c.GetThreadObservationRevision(f.ctx, f.threadID, f.thread.SourceUpdatedAt, f.thread.ObservationSequence)
+		if err != nil {
+			return err
+		}
+		refs = []corpus.ObservationRef{{Kind: "thread", ID: observation.ID}}
+	} else {
+		observations, _, err := f.c.ListFacetObservationsBounded(f.ctx, f.repoID, &f.threadID, FacetIssueTimeline, 100)
+		if err != nil {
+			return err
+		}
+		for _, observation := range observations {
+			var page []github.IssueTimelineEvent
+			if err := json.Unmarshal([]byte(observation.Payload), &page); err != nil {
+				return fmt.Errorf("decode issue timeline provenance: %w", err)
+			}
+			for _, event := range page {
+				if event.Event == "closed" && event.CommitID == selectedCommit {
+					refs = append(refs, corpus.ObservationRef{Kind: "facet", ID: observation.ID})
+					break
+				}
+			}
+		}
+		if len(refs) == 0 {
+			return errors.New("closing commit timeline observation is unavailable")
+		}
+	}
+	_, err := f.c.SaveResolutionRecord(f.ctx, corpus.ResolutionRecord{ThreadID: f.threadID, Kind: kind, Summary: summary, RuleVersion: "resolution.v1", SourceUpdatedAt: sourceUpdatedAt, SourceObservationRefs: refs})
+	return err
+}
+
 type facetRunner struct {
 	ctx      context.Context
 	c        *corpus.Corpus
@@ -289,7 +414,13 @@ func (f *facetRunner) hydrateIssueComments() (HydratedFacet, error) {
 	if err := f.ctx.Err(); err != nil {
 		return HydratedFacet{}, err
 	}
-	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetIssueComments, sourceUpdatedAt, pageObservations, complete, f.runID); err != nil {
+	if !complete {
+		if err := f.c.AdvanceFacet(f.ctx, f.repoID, &f.threadID, FacetIssueComments, sourceUpdatedAt, false, f.runID); err != nil {
+			return HydratedFacet{}, err
+		}
+		return HydratedFacet{Facet: FacetIssueComments, Count: total, Pages: pages, Complete: false}, nil
+	}
+	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetIssueComments, sourceUpdatedAt, pageObservations, true, f.runID); err != nil {
 		return HydratedFacet{}, err
 	}
 
@@ -367,7 +498,13 @@ func (f *facetRunner) hydratePullRequestReviews() (HydratedFacet, error) {
 	if err := f.ctx.Err(); err != nil {
 		return HydratedFacet{}, err
 	}
-	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetPRReviews, sourceUpdatedAt, pageObservations, complete, f.runID); err != nil {
+	if !complete {
+		if err := f.c.AdvanceFacet(f.ctx, f.repoID, &f.threadID, FacetPRReviews, sourceUpdatedAt, false, f.runID); err != nil {
+			return HydratedFacet{}, err
+		}
+		return HydratedFacet{Facet: FacetPRReviews, Count: total, Pages: pages, Complete: false}, nil
+	}
+	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetPRReviews, sourceUpdatedAt, pageObservations, true, f.runID); err != nil {
 		return HydratedFacet{}, err
 	}
 
@@ -422,7 +559,13 @@ func (f *facetRunner) hydratePullRequestReviewComments() (HydratedFacet, error)
 	if err := f.ctx.Err(); err != nil {
 		return HydratedFacet{}, err
 	}
-	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetPRReviewComments, sourceUpdatedAt, pageObservations, complete, f.runID); err != nil {
+	if !complete {
+		if err := f.c.AdvanceFacet(f.ctx, f.repoID, &f.threadID, FacetPRReviewComments, sourceUpdatedAt, false, f.runID); err != nil {
+			return HydratedFacet{}, err
+		}
+		return HydratedFacet{Facet: FacetPRReviewComments, Count: total, Pages: pages, Complete: false}, nil
+	}
+	if err := f.c.ApplyFacetObservationSet(f.ctx, f.repoID, &f.threadID, FacetPRReviewComments, sourceUpdatedAt, pageObservations, true, f.runID); err != nil {
 		return HydratedFacet{}, err
 	}