fix(mcp): steer read_scope vs search by scope size so the client self-selects (#131)

## Problem

When testing the MCP connect flow against `app-dev.vana.org`, Claude
reliably **hung on "Working"** after issuing `read_scope` on large
scopes (`spotify.playlists` 546 KB, `chatgpt.conversations` 1.6 MB),
while small scopes (`linkedin.profile` 39 KB) returned instantly. The
reads *succeeded* on the PS side — but a full read of a large scope
comes back as many 256 KB pages the client must walk via `nextCursor`,
and that pagination loop stalls the agent.

The client had **no reliable signal** to choose read vs search, and the
signal it did have was actively misleading: `list_granted_scopes`
emitted `searchRecommended: false` for large/huge scopes. Under that
field name a model reads it as *"search not recommended → read the whole
thing"* — i.e. it steers toward the exact slow path. (The flag actually
meant "eligible for un-named broad search," not "should you search
this.")

The goal: the client should pick the right tool **by itself**, no
operator hand-holding.

## Changes

- **Directive routing field.** `isSearchRecommended` →
`recommendAccessForScope`, emitting `recommendedAccess: "read" |
"search" | "wait"` per scope:
  - tiny/small/medium ready → `read`
- large/huge / unknown size → `search` (+ reason: name the scope
explicitly; a full read returns many pages via nextCursor)
  - still indexing → `wait`
- **In-band steer at point of action.** `read_scope` now fetches
lightweight index-only metadata (best-effort, never blocks the read) and
attaches a `guidance` hint **only when the scope is large enough that
search is better** — so even if the model skips `list_granted_scopes`,
the first page tells it to pivot. Small reads get no extra noise.
- **Sharper tool descriptions.** `list_granted_scopes` ("Call first,
then follow recommendedAccess"); `read_scope` ("best for
tiny/small/medium; for large/huge prefer search_personal_context");
`search_personal_context` ("preferred way to pull from large/huge
scopes… name scopes explicitly").

## Scope / non-goals

This is a tool-definition + advisory-signal fix. It does **not** address
the deeper pagination-completion issue (large-scope `nextCursor` loops
through the browser PS); that's runtime/storage and tracked separately.

## Testing

- `npx vitest run packages/core/src/mcp/tools.test.ts
packages/core/src/mcp/read-client.test.ts` → **22/22 pass** (updated
`list_granted_scopes` expectations to `recommendedAccess`; added
coverage for the `read_scope` guidance steer — present for large scopes,
absent for small).
- `npm run build`, `npm run lint` (tsc --noEmit), `npm run format:check`
→ clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Tim Nunamaker <tnunamak@gmail.com>

Author: 3 people

packages/core/src/mcp/read-client.ts

@@ -53,7 +53,7 @@ export interface McpDataReadClient {
   /**
    * Return lightweight planning metadata for a scope using the storage index
    * only — no network call, no block read, no auth. Used by list_granted_scopes
-   * to populate sizeBytes/sizeClass/searchRecommended without reading data.
+   * to populate sizeBytes/sizeClass/recommendedAccess without reading data.
    * Returns null when the scope has no local index entry.
    */
   getScopeMetadata(scope: string): Promise<McpScopeMetadata | null>;

packages/core/src/mcp/tools.test.ts

@@ -245,6 +245,103 @@ describe("mcp/tools", () => {
     });
   });
 
+  it("read_scope attaches search guidance for large scopes and omits it for small ones", async () => {
+    const block = {
+      id: "b1",
+      path: "$.items[0:1]",
+      mediaType: "application/json",
+      value: { id: 1 },
+      sizeBytes: 20,
+    };
+    const makeClient = (sizeBytes: number) =>
+      createMinimalReadClient({
+        getScopeMetadata: vi.fn(async (scope: string) => ({
+          scope,
+          collectedAt: "2026-06-05T00:00:00Z",
+          sizeBytes,
+          hasBlocks: true,
+        })),
+        readScopeBlocks: vi.fn().mockResolvedValue({
+          scope: "instagram.profile",
+          collectedAt: "2026-06-05T00:00:00Z",
+          contentKind: "json",
+          blocks: [block],
+          nextCursor: "cursor-2",
+          warnings: [],
+        }),
+      });
+
+    // Large scope (~5MB) → guidance steering to search, read still served.
+    const large = await getTool("read_scope").handler(
+      { scope: "instagram.profile" },
+      {
+        connection: createConnection(),
+        readClient: makeClient(5_000_000) as never,
+      },
+    );
+    const largePayload = JSON.parse(large.content[0].text);
+    expect(largePayload.blocks).toHaveLength(1);
+    expect(largePayload.guidance).toMatchObject({
+      recommendedAccess: "search",
+      sizeClass: "large",
+    });
+
+    // Small scope (5KB) → no guidance noise.
+    const small = await getTool("read_scope").handler(
+      { scope: "instagram.profile" },
+      {
+        connection: createConnection(),
+        readClient: makeClient(5_000) as never,
+      },
+    );
+    expect(JSON.parse(small.content[0].text).guidance).toBeUndefined();
+  });
+
+  it("read_scope does not wait for stalled metadata guidance", async () => {
+    const readClient = createMinimalReadClient({
+      getScopeMetadata: vi.fn(
+        () =>
+          new Promise(() => {
+            // Deliberately never resolves: guidance is advisory only.
+          }),
+      ),
+      readScopeBlocks: vi.fn().mockResolvedValue({
+        scope: "instagram.profile",
+        collectedAt: "2026-06-05T00:00:00Z",
+        contentKind: "json",
+        blocks: [
+          {
+            id: "b1",
+            path: "$.profile",
+            mediaType: "application/json",
+            value: { username: "tim" },
+            sizeBytes: 18,
+          },
+        ],
+        warnings: [],
+      }),
+    });
+
+    const started = Date.now();
+    const result = await getTool("read_scope").handler(
+      { scope: "instagram.profile", timeoutMs: 1000 },
+      {
+        connection: createConnection(),
+        readClient: readClient as never,
+      },
+    );
+    const elapsed = Date.now() - started;
+
+    expect(elapsed).toBeLessThan(500);
+    expect(result.isError).toBeUndefined();
+    expect(JSON.parse(result.content[0].text)).toMatchObject({
+      scope: "instagram.profile",
+      page: {
+        returnedBlocks: 1,
+      },
+    });
+  });
+
   it("read_scope returns a typed timeout instead of hanging on a stalled read", async () => {
     const readClient = createMinimalReadClient({
       readScopeBlocks: vi.fn(
@@ -743,15 +840,15 @@ describe("mcp/tools", () => {
       payload.scopes.map((s: { scope: string }) => [s.scope, s]),
     );
 
-    // instagram.profile: 5KB → tiny → searchRecommended: true
+    // instagram.profile: 5KB → tiny → read the whole scope directly
     expect(byScope["instagram.profile"].dataStatus).toBe("ready");
-    expect(byScope["instagram.profile"].searchRecommended).toBe(true);
+    expect(byScope["instagram.profile"].recommendedAccess).toBe("read");
     expect(byScope["instagram.profile"].sizeBytes).toBe(5_000);
     expect(byScope["instagram.profile"].sizeClass).toBe("tiny");
 
-    // chatgpt.history: 15MB → huge → searchRecommended: false
+    // chatgpt.history: 15MB → huge → steer to search instead of a full read
     expect(byScope["chatgpt.history"].dataStatus).toBe("ready");
-    expect(byScope["chatgpt.history"].searchRecommended).toBe(false);
+    expect(byScope["chatgpt.history"].recommendedAccess).toBe("search");
     expect(byScope["chatgpt.history"].sizeClass).toBe("huge");
     expect(byScope["chatgpt.history"].reason).toBeDefined();
   });
@@ -769,7 +866,7 @@ describe("mcp/tools", () => {
     const payload = JSON.parse(result.content[0].text);
     for (const scope of payload.scopes) {
       expect(scope.dataStatus).toBe("needs_refresh");
-      expect(scope.searchRecommended).toBe(false);
+      expect(scope.recommendedAccess).toBe("wait");
     }
   });
 
@@ -801,7 +898,7 @@ describe("mcp/tools", () => {
       (scope: { scope: string }) => scope.scope === "instagram.profile",
     );
     expect(profile.dataStatus).toBe("indexing");
-    expect(profile.searchRecommended).toBe(false);
+    expect(profile.recommendedAccess).toBe("wait");
     expect(profile.reason).toContain("indexing");
   });
 
@@ -836,7 +933,7 @@ describe("mcp/tools", () => {
       expect.objectContaining({
         scope: "instagram.profile",
         dataStatus: "indexing",
-        searchRecommended: false,
+        recommendedAccess: "wait",
       }),
     ]);
   });

packages/core/src/mcp/tools.ts

@@ -359,30 +359,42 @@ function classifySizeBytes(sizeBytes: number | undefined): SizeClass {
   return "huge";
 }
 
-function isSearchRecommended(
+type RecommendedAccess = "read" | "search" | "wait";
+
+// Directive guidance the MCP client (e.g. Claude) uses to pick read_scope vs
+// search_personal_context for a scope WITHOUT operator intervention. The signal
+// must be unambiguous: a misnamed flag here (the old `searchRecommended: false`
+// on large scopes) reads to a model as "don't search → read the whole thing",
+// which is exactly the slow path — a full read of a large scope returns many
+// pages it must walk via nextCursor and tends to stall the agent loop.
+function recommendAccessForScope(
   sizeClass: SizeClass,
   hasBlocks: boolean,
-): { recommended: boolean; reason?: string } {
+): { recommendedAccess: RecommendedAccess; reason: string } {
   if (!hasBlocks) {
     return {
-      recommended: false,
+      recommendedAccess: "wait",
       reason:
-        "bounded block data is still indexing or unavailable; retry shortly",
+        "bounded block data is still indexing or unavailable; retry shortly before reading or searching",
     };
   }
   if (sizeClass === "large" || sizeClass === "huge") {
     return {
-      recommended: false,
-      reason: `scope is ${sizeClass}; use explicit scopes with cursor to search safely`,
+      recommendedAccess: "search",
+      reason: `scope is ${sizeClass}; prefer search_personal_context and name this scope explicitly. A full read_scope returns many pages you must walk via nextCursor.`,
     };
   }
   if (sizeClass === "unknown") {
     return {
-      recommended: false,
-      reason: "scope size unknown; pass explicitly to search",
+      recommendedAccess: "search",
+      reason:
+        "scope size is unknown; prefer search_personal_context and name this scope explicitly",
     };
   }
-  return { recommended: true };
+  return {
+    recommendedAccess: "read",
+    reason: `scope is ${sizeClass}; read_scope returns it in one or few pages. search_personal_context also works for targeted lookups.`,
+  };
 }
 
 async function resolveSearchScopes({
@@ -523,7 +535,7 @@ const listGrantedScopes: McpToolDefinition = {
   name: "list_granted_scopes",
   title: "List granted scopes",
   description:
-    "List granted scopes with dataStatus, sizeClass, searchRecommended, and sizeBytes. Call first.",
+    "List granted scopes with dataStatus, sizeClass, sizeBytes, and recommendedAccess ('read' for small scopes, 'search' for large/huge scopes, 'wait' while indexing). Call first, then follow recommendedAccess.",
   inputSchema: {},
   async handler(_args, { connection, readClient }) {
     const grantedScopes = uniqueScopes(connection);
@@ -543,7 +555,7 @@ const listGrantedScopes: McpToolDefinition = {
               source: scope.split(".")[0],
               dataStatus: "indexing" as const,
               sizeClass: "unknown" as SizeClass,
-              searchRecommended: false,
+              recommendedAccess: "wait" as RecommendedAccess,
               reason:
                 "scope metadata is still indexing or temporarily unavailable; retry shortly",
             };
@@ -554,12 +566,12 @@ const listGrantedScopes: McpToolDefinition = {
             scope,
             dataStatus: "needs_refresh" as const,
             sizeClass: "unknown" as SizeClass,
-            searchRecommended: false,
+            recommendedAccess: "wait" as RecommendedAccess,
             reason: "no local data found; refresh your data connection",
           };
         }
         const sizeClass = classifySizeBytes(meta.sizeBytes);
-        const { recommended, reason } = isSearchRecommended(
+        const { recommendedAccess, reason } = recommendAccessForScope(
           sizeClass,
           meta.hasBlocks,
         );
@@ -572,8 +584,8 @@ const listGrantedScopes: McpToolDefinition = {
             : ("indexing" as const),
           sizeBytes: meta.sizeBytes,
           sizeClass,
-          searchRecommended: recommended,
-          ...(reason ? { reason } : {}),
+          recommendedAccess,
+          reason,
         };
       }),
     );
@@ -585,7 +597,7 @@ const readScope: McpToolDefinition = {
   name: "read_scope",
   title: "Read scope",
   description:
-    "Read one approved scope as bounded blocks. Page with nextCursor for large scopes.",
+    "Read one full approved scope as bounded blocks. Best for tiny/small/medium scopes (see sizeClass from list_granted_scopes). For large/huge scopes prefer search_personal_context — a full read returns many pages you must walk via nextCursor and is slow. The response includes a `guidance` hint when a scope is large enough that search is the better tool.",
   inputSchema: {
     scope: z
       .string()
@@ -639,6 +651,49 @@ const readScope: McpToolDefinition = {
       1000,
       MAX_READ_SCOPE_TIMEOUT_MS,
     );
+
+    // In-band steer: the model may call read_scope on a large scope without
+    // first consulting list_granted_scopes. Fetch lightweight planning metadata
+    // (storage-index only, no network/auth) opportunistically so we can attach a
+    // `guidance` hint when it is already available. This is advisory only: it
+    // must never delay the read or consume the read timeout budget.
+    let guidance:
+      | {
+          sizeClass: SizeClass;
+          sizeBytes?: number;
+          recommendedAccess: RecommendedAccess;
+          reason: string;
+        }
+      | undefined;
+    if (typeof readClient.getScopeMetadata === "function") {
+      void withTimeout(
+        readClient.getScopeMetadata(scope),
+        DEFAULT_SCOPE_METADATA_TIMEOUT_MS,
+        `scope metadata for ${scope}`,
+      )
+        .then((meta) => {
+          if (!meta) return;
+          const sizeClass = classifySizeBytes(meta.sizeBytes);
+          const recommendation = recommendAccessForScope(
+            sizeClass,
+            meta.hasBlocks,
+          );
+          // Only steer when search is the better tool; a normal small read gets
+          // no extra noise.
+          if (recommendation.recommendedAccess === "search") {
+            guidance = {
+              sizeClass,
+              sizeBytes: meta.sizeBytes,
+              recommendedAccess: recommendation.recommendedAccess,
+              reason: recommendation.reason,
+            };
+          }
+        })
+        .catch(() => {
+          // Advisory only — serve the read without guidance.
+        });
+    }
+
     try {
       const result = await withTimeout(
         readClient.readScopeBlocks({
@@ -658,6 +713,7 @@ const readScope: McpToolDefinition = {
         contentKind: result.contentKind,
         blocks: result.blocks,
         ...(result.nextCursor ? { nextCursor: result.nextCursor } : {}),
+        ...(guidance ? { guidance } : {}),
         warnings: result.warnings,
         page: {
           cursor: cursor ?? null,
@@ -762,7 +818,7 @@ const searchPersonalContext: McpToolDefinition = {
   name: "search_personal_context",
   title: "Search personal context",
   description:
-    "Search approved scopes. Omit scopes for small ready data; pass scopes to target. Continue with nextSearchCursor.",
+    "Search approved scopes by keyword — the preferred way to pull from large/huge scopes without reading every page. Omit scopes to sweep all small ready data; name scopes (including large ones) to target them. Continue with nextSearchCursor.",
   inputSchema: {
     query: z.string().min(1).max(SEARCH_QUERY_MAX_CHARS),
     scopes: z