Skip to content

fix(deriver): reduce ephemeral explicit observations#445

Open
Vein05 wants to merge 2 commits intoplastic-labs:mainfrom
Vein05:fix/444-deriver-ephemeral-conclusions
Open

fix(deriver): reduce ephemeral explicit observations#445
Vein05 wants to merge 2 commits intoplastic-labs:mainfrom
Vein05:fix/444-deriver-ephemeral-conclusions

Conversation

@Vein05
Copy link
Copy Markdown

@Vein05 Vein05 commented Mar 24, 2026

Summary

  • narrow the minimal deriver prompt toward durable, cross-session-useful observations
  • explicitly exclude one-off tasks, transient workflow updates, and short-lived session logistics from explicit observation extraction
  • add a focused prompt regression test covering the new durable vs ephemeral guidance

Closes #444

Validation

  • uv run ruff check src/deriver/prompts.py tests/deriver/test_prompts.py tests/deriver/test_representation_crud.py
  • uv run pytest --noconftest -n0 tests/deriver/test_prompts.py tests/deriver/test_representation_crud.py -q
  • uv run pytest -n0 tests/deriver -q

All tests passed.

Notes

  • this change is intentionally prompt-first and does not add heuristic filtering to the deriver pipeline, as that would require bigger changes
  • the codebase currently doesn't include a dedicated benchmark for ephemeral-observation suppression vs durable-memory retention, so validation here is minimal.

Summary by CodeRabbit

  • Bug Fixes

    • Tightened observation filtering to prioritize durable facts (standing instructions, long-lived preferences, relationships, commitments, ongoing situations) and exclude transient items (one-off tasks, temporary workflow steps, ephemeral process updates).
    • Added explicit handling for declared enduring preferences (e.g., “short version first”).
  • Tests

    • Added test coverage verifying durable-observation bias, explicit skip examples for transient items, and respect for declared preferences.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 24, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b21fc271-e18f-45a6-8a8a-1698f5ac07b2

📥 Commits

Reviewing files that changed from the base of the PR and between 4c60eb1 and 7e4cf01.

📒 Files selected for processing (1)
  • tests/deriver/test_prompts.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/deriver/test_prompts.py

Walkthrough

Modified the minimal deriver prompt to extract only durable, long-lived observations (standing instructions, long-lived preferences, relationships, commitments, ongoing situations) and to exclude one-off tasks, temporary workflow steps, and ephemeral session updates unless explicitly stated as enduring. Added corresponding test validating the prompt text.

Changes

Cohort / File(s) Summary
Prompt Refinement
src/deriver/prompts.py
Tightened minimal_deriver_prompt instructions to prioritize durable facts and skip transient/session-ephemeral items; updated examples to include a durable-preference example and SKIP examples for transient actions.
Test Coverage
tests/deriver/test_prompts.py
New test that dynamically imports the prompts module and asserts minimal_deriver_prompt(peer_id="alice", ...) contains directives favoring durable observations, explicit SKIP entries for transient phrases, and an EXPLICIT echo of the input request.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 I hop the prompts with careful paws,

keeping what lasts and hiding the claws.
Short first, I whisper, let habits stay—
fleeting tasks, you skip away.
✨🧺

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(deriver): reduce ephemeral explicit observations' directly summarizes the main change: narrowing the minimal deriver prompt to exclude ephemeral/transient observations and favor durable ones.
Linked Issues check ✅ Passed The changes implement the core objective from #444 by modifying the minimal_deriver_prompt to explicitly exclude session-ephemeral events (temporary tasks, workflow steps, session logistics) and prioritize durable observations. The test validates this behavior with specific assertions on prompt content.
Out of Scope Changes check ✅ Passed All changes are scoped to the deriver prompt logic and its test coverage. No out-of-scope modifications to unrelated systems, filtering mechanisms, or other modules were introduced.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/deriver/test_prompts.py (1)

1-12: Replace dynamic module loading with standard imports for simplicity.

The importlib pattern here adds unnecessary complexity. Since the project already imports from src.* throughout the test suite, a standard import would be simpler and more maintainable:

from src.deriver.prompts import minimal_deriver_prompt

Additionally, the bare assert on line 8 will produce a cryptic ImportError or AssertionError at test collection time rather than a clear test failure if something goes wrong.

♻️ Proposed simplification
-import importlib.util
-from pathlib import Path
-
-_PROMPTS_PATH = Path(__file__).resolve().parents[2] / "src" / "deriver" / "prompts.py"
-_PROMPTS_SPEC = importlib.util.spec_from_file_location(
-    "deriver_prompts", _PROMPTS_PATH
-)
-assert _PROMPTS_SPEC is not None and _PROMPTS_SPEC.loader is not None
-_PROMPTS_MODULE = importlib.util.module_from_spec(_PROMPTS_SPEC)
-_PROMPTS_SPEC.loader.exec_module(_PROMPTS_MODULE)
-
-minimal_deriver_prompt = _PROMPTS_MODULE.minimal_deriver_prompt
+from src.deriver.prompts import minimal_deriver_prompt
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/deriver/test_prompts.py` around lines 1 - 12, Replace the dynamic
import block (_PROMPTS_PATH, _PROMPTS_SPEC, _PROMPTS_MODULE and the assert +
exec_module calls) with a normal import statement that directly imports
minimal_deriver_prompt from src.deriver.prompts (e.g. from src.deriver.prompts
import minimal_deriver_prompt); remove the manual spec/loader assertions and
exec_module usage so import errors surface as normal ImportError during test
collection instead of relying on the bare assert.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/deriver/test_prompts.py`:
- Around line 1-12: Replace the dynamic import block (_PROMPTS_PATH,
_PROMPTS_SPEC, _PROMPTS_MODULE and the assert + exec_module calls) with a normal
import statement that directly imports minimal_deriver_prompt from
src.deriver.prompts (e.g. from src.deriver.prompts import
minimal_deriver_prompt); remove the manual spec/loader assertions and
exec_module usage so import errors surface as normal ImportError during test
collection instead of relying on the bare assert.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 46b7290d-f7c0-499e-90d7-586dbd0271b8

📥 Commits

Reviewing files that changed from the base of the PR and between b3c1917 and 4c60eb1.

📒 Files selected for processing (2)
  • src/deriver/prompts.py
  • tests/deriver/test_prompts.py

@Vein05
Copy link
Copy Markdown
Author

Vein05 commented Mar 24, 2026

@VVoruganti what would you suggest here?

I'm getting a

ValueError: Missing client for Summary: google

when i try to import from src it through src.deriver.__init__, which turns into app/client bootstrap and fails. Do we need to load the entire client for this simple test. If needed, I can work around that!

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deriver creates ephemeral session events as permanent conclusions

1 participant