Skip to content

[codex] Improve Slackbot personalization and resilience#1338

Merged
zzstoatzz merged 4 commits into
mainfrom
codex/slackbot-personalization
Apr 22, 2026
Merged

[codex] Improve Slackbot personalization and resilience#1338
zzstoatzz merged 4 commits into
mainfrom
codex/slackbot-personalization

Conversation

@zzstoatzz

Copy link
Copy Markdown
Collaborator

What changed

This updates the Slackbot along three related axes:

  • makes GitHub issue/discussion lookup degrade cleanly on auth and rate-limit failures instead of failing the whole turn
  • improves user personalization by turning TurboPuffer user facts into a more explicit prompt block with seen-before state, recurring profile context, relevant prior notes, and conflict caveats
  • refreshes and clarifies model configuration so the main response model, bot override model, and memory-synthesis model are easier to inspect and control

Why

Marvin had useful personalization data in TurboPuffer, but it was being injected as a thin raw notes blob. That made it harder for the bot to answer like it actually remembers the person, and it also let contradictory stored facts leak into prompt context too easily.

Separately, expired GitHub credentials were causing issue-search failures to bubble up too aggressively. The Slackbot now degrades more safely while still surfacing the outage.

Impact

  • returning users should get more personalized answers grounded in recurring context and relevant prior questions
  • contradictory stored memory is treated as uncertain instead of being injected as truth
  • Prefect runs now log non-sensitive model settings for easier diagnosis
  • marvin_bot_model now actually works as an override again
  • marvin_memory_synthesis_model can be configured via Prefect Variable, alongside the other model settings

Validation

Ran:

MARVIN_SLACKBOT_SLACK_API_TOKEN=test-token uv run --python 3.13 --extra slackbot pytest tests/slackbot/test_personalization.py tests/slackbot/test_prompting.py tests/slackbot/test_search.py

All 13 tests passed.

@zzstoatzz zzstoatzz marked this pull request as ready for review April 22, 2026 16:58
Copilot AI review requested due to automatic review settings April 22, 2026 16:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the Slackbot’s resilience to GitHub lookup failures, introduces a more structured personalization block derived from TurboPuffer “user facts,” and clarifies model configuration (response vs. memory-synthesis) via Prefect Variables.

Changes:

  • Add explicit personalization fields to UserContext and generate a structured system-prompt personalization section.
  • Make GitHub issue lookup degrade gracefully on auth / rate-limit / not-found errors.
  • Split model configuration into marvin_ai_model, marvin_bot_model, and marvin_memory_synthesis_model, and log model settings during runs.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
examples/slackbot/src/slackbot/types.py Expands UserContext to include personalization fields.
examples/slackbot/src/slackbot/settings.py Adds separate settings for response model vs. memory-synthesis model.
examples/slackbot/src/slackbot/search.py Wraps GitHub issue search with targeted error handling.
examples/slackbot/src/slackbot/core.py Loads a personalization snapshot and centralizes system prompt construction.
examples/slackbot/src/slackbot/api.py Logs agent/model configuration and uses the updated agent creation path.
examples/slackbot/src/slackbot/_internal/prompting.py New helper to build structured system prompts with personalization sections.
examples/slackbot/src/slackbot/_internal/personalization.py New personalization snapshot synthesis (fact selection + optional summarization).
examples/slackbot/README.md Documents the new model variables and updated defaults.
Comments suppressed due to low confidence (1)

examples/slackbot/src/slackbot/core.py:178

  • create_agent() always wraps settings.bot_model_name in AnthropicModel(...). If marvin_ai_model/marvin_bot_model is set to an OpenAI model name like gpt-5 (as the README advertises), this will try to run it through the Anthropic client and fail. Consider either passing the model name directly to Agent(model=...) (letting pydantic-ai select the provider), or branching to construct the correct model/provider based on the selected model name.
    ai_model = model or AnthropicModel(
        model_name=settings.bot_model_name,
        provider=Provider(
            api_key=Secret.load(settings.anthropic_key_secret_name, _sync=True).get(),  # type: ignore
        ),
    )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread examples/slackbot/src/slackbot/core.py Outdated
Comment on lines 145 to 152
namespace = f"{settings.user_facts_namespace_prefix}{user_id}"
personalization = load_personalization_snapshot(namespace, user_question)
try:
user_notes = query_namespace(
query_text=user_question,
namespace=f"{settings.user_facts_namespace_prefix}{user_id}",
namespace=namespace,
top_k=5,
)
Comment thread examples/slackbot/src/slackbot/api.py Outdated
Comment on lines +99 to +105
"Agent config: response_model=%s memory_synthesis_model=%s temperature=%s max_tool_calls=%s seen_before=%s workspace=%s",
settings.bot_model_name,
settings.memory_synthesis_model_name,
settings.temperature,
settings.max_tool_calls_per_turn,
user_context["seen_before"],
user_context["workspace_name"],
Comment on lines +196 to +197
except GitHubError as exc:
return f"GitHub issue search failed: {exc}"
@zzstoatzz zzstoatzz merged commit 5c66f49 into main Apr 22, 2026
3 checks passed
@zzstoatzz zzstoatzz deleted the codex/slackbot-personalization branch April 22, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants