Skip to content

Commit 83901e1

Browse files
Alex-Hunterzclaude
andcommitted
fix: scope instr3/4, temporal notes use event_time_only (no created_at fallback)
- instr 3: narrow trigger to explicit list questions; add "recently/latest" → single-match carve-out - instr 4: revert to absolute-date preference (relative-phrasing experiment degraded answers) - _format_retrieved_context: _relative_time_note now uses _event_time_only for all item types — prevents April-2026 hallucinations when event_time missing (syntheses main case) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent ca74845 commit 83901e1

2 files changed

Lines changed: 9 additions & 8 deletions

File tree

benchmarks/locomo/locomo_runner.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -934,7 +934,7 @@ def _format_retrieved_context(search_results: Any) -> str:
934934
date = _date_prefix(timestamp)
935935
date_prefix = f"[{date}] " if date else ""
936936
text = str(fact.get("text", str(fact))).strip()
937-
text = f"{text}{_relative_time_note(text, timestamp)}"
937+
text = f"{text}{_relative_time_note(text, _event_time_only(fact))}"
938938
lines.append(f"{i}. {date_prefix}{text}")
939939

940940
episodes = search_results.get("episodes") or []
@@ -945,7 +945,7 @@ def _format_retrieved_context(search_results: Any) -> str:
945945
date = _date_prefix(timestamp)
946946
date_prefix = f"[{date}] " if date else ""
947947
text = str(ep.get("text", str(ep))).strip()
948-
text = f"{text}{_relative_time_note(text, timestamp)}"
948+
text = f"{text}{_relative_time_note(text, _event_time_only(ep))}"
949949
lines.append(f"{i}. {date_prefix}{text}")
950950

951951
syntheses = search_results.get("syntheses") or []
@@ -956,7 +956,7 @@ def _format_retrieved_context(search_results: Any) -> str:
956956
date = _date_prefix(timestamp)
957957
date_prefix = f"[{date}] " if date else ""
958958
text = str(sy.get("text", str(sy))).strip()
959-
text = f"{text}{_relative_time_note(text, timestamp)}"
959+
text = f"{text}{_relative_time_note(text, _event_time_only(sy))}"
960960
lines.append(f"{i}. {date_prefix}{text}")
961961

962962
sentences = search_results.get("sentences") or []
@@ -986,7 +986,7 @@ def _format_retrieved_context(search_results: Any) -> str:
986986
prefix = f"[{role.upper()}] " if role else ""
987987
timestamp = _timestamp_for_context(sent)
988988
text = str(sent.get("text", str(sent))).strip()
989-
text = f"{text}{_relative_time_note(text, timestamp)}"
989+
text = f"{text}{_relative_time_note(text, _event_time_only(sent))}"
990990
lines.append(f" {prefix}{text}")
991991

992992
return "\n".join(lines) if lines else "No relevant context retrieved."

vektori/qa/generator.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,15 +26,16 @@ async def generate(self, prompt: str, max_tokens: int | None = None) -> str:
2626
INSTRUCTIONS:
2727
1. Use only the provided context. Logical inference and reasoning from context facts is expected and correct — this is NOT guessing. Do not introduce facts that have no support in any context item.
2828
2. First, silently find every fact, episode, synthesis, or transcript line that directly relates to the question. Do not print this evidence list.
29-
3. For counting, frequency, list, "all", "total", aggregation, or "both X and Y" questions:
29+
3. For questions explicitly asking for a list, set, or all items ("all", "both", "how many", "what types", "what events", "ways", "which of"):
3030
- Do not stop after the first match. Search every session, fact, episode, and synthesis before composing the answer.
3131
- Honour the full scope of the question: "family and friends" means both; "events" means all events, not just the most prominent one.
3232
- If you find N items, verify there are no more before finalising. Return the complete set as a concise list.
3333
- Deduplicate only when the context clearly describes the same event more than once.
34+
- For questions asking about "recently", "lately", or "the latest" — return only the single most recent match, not a full list.
3435
4. For date, time, order, recency, or "when" questions:
35-
- If the context uses a relative expression to describe when something happened (e.g. "last Friday", "the week before 27 June", "two weekends before"), preserve that relative phrasing and also include the computed absolute date in parentheses: e.g. "the week before 27 June 2023 (approx. 19–25 June 2023)".
36-
- For direct date facts stated as absolute dates, use the absolute date.
37-
- If the context includes a temporal note, use it to resolve relative wording.
36+
- Use exact absolute dates from the context whenever available.
37+
- Do not answer with relative time words like "recently" or "lately" when an absolute date is available.
38+
- If the context includes a temporal note, use it to resolve relative wording into an absolute date.
3839
5. For changed or updated information:
3940
- Prefer the most recent value when later context overrides earlier context.
4041
- Mention older values only if the question asks for history or change over time.

0 commit comments

Comments
 (0)