Skip to content

Capture: smart existing entry selection for LLM prompt #89

Description

@itdove

Summary

extract_knowledge sends existing entries to the LLM so it can avoid duplicates. Currently sends all keys (truncated to last 50). For large knowledge bases, this wastes prompt tokens on irrelevant entries.

Current behavior

existing_text = "\n".join(f"- {e.key}" for e in existing[-50:])

Proposed improvement

Use vector search to find existing entries most similar to the transcript, then only include those in the prompt:

  1. Embed the transcript (or a summary of it)
  2. query_vector(embedding, limit=20) to get closest existing entries
  3. Send only those 20 entries (keys + short value snippets) to the LLM

This gives the LLM better context for dedup while keeping the prompt small.

Per-model token budgets (future)

Different models have different context windows. Could add per-model config:

  • phi4-mini: ~4K usable for existing entries
  • llama3: ~8K
  • claude: ~100K

The capture prompt would auto-adjust the number of existing entries based on model capacity.

Acceptance Criteria

  • Existing entries sent to capture LLM are filtered by relevance to transcript
  • Prompt stays within model capacity
  • Capture still detects duplicates effectively
  • Falls back to keys-only if embedding unavailable

Discovered during #37 demo testing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions