fix: QA prompt + judge fixes for locomo score improvement

- instr 3: exhaustive list enumeration, honour full question scope
- instr 4: preserve relative date phrasing + add absolute in parens
- instr 5: milestone event priority for significant-event questions
- instr 9: strict person attribution — verify name before using fact
- instr 10 (new): match answer length to question scope
- judge: relative=absolute date equivalence rule + arithmetic check
- judge: add missing abstention phrase variants (any/no information)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: Alex-Hunterz

benchmarks/locomo/locomo_judge.py

@@ -29,12 +29,15 @@
 ABSTENTION_PHRASES = (
     "i don't have that information",
     "i do not have that information",
+    "i don't have any information",
+    "i do not have any information",
     "i don't have enough information",
     "i do not have enough information",
     "i cannot answer",
     "i can't answer",
     "cannot answer this",
     "not enough information",
+    "no information in the provided context",
     "information not available",
     "i don't know",
     "i do not know",
@@ -69,6 +72,10 @@
 - For date answers, accept equivalent formats. These are always CORRECT:
   "2023-10-21" = "21 October 2023" = "October 21, 2023"
   "2022-11-10" = "10 November 2022" = "November 10, 2022"
+- Absolute dates that equal a relative expression are CORRECT. Verify the arithmetic before marking wrong:
+  "July 14, 2023" = "The Friday before 15 July 2023" (July 14 is a Friday)
+  "11 August 2023" = "The Friday before 14 August 2023" (August 11 is a Friday)
+  A model that gives the computed absolute date when the expected uses relative phrasing is CORRECT, not wrong.
 - Extra correct details do not make an answer wrong.
 
 JSON schema:

vektori/qa/generator.py

@@ -26,21 +26,24 @@ async def generate(self, prompt: str, max_tokens: int | None = None) -> str:
 INSTRUCTIONS:
 1. Use only the provided context. Logical inference and reasoning from context facts is expected and correct — this is NOT guessing. Do not introduce facts that have no support in any context item.
 2. First, silently find every fact, episode, synthesis, or transcript line that directly relates to the question. Do not print this evidence list.
-3. For counting, frequency, list, "all", "total", or aggregation questions:
-   - Consider every relevant item across all sessions, not just the first match.
-   - Deduplicate only if the context clearly describes the same item more than once.
-   - Return the complete set or total. A compact calculation is allowed when it helps.
+3. For counting, frequency, list, "all", "total", aggregation, or "both X and Y" questions:
+   - Do not stop after the first match. Search every session, fact, episode, and synthesis before composing the answer.
+   - Honour the full scope of the question: "family and friends" means both; "events" means all events, not just the most prominent one.
+   - If you find N items, verify there are no more before finalising. Return the complete set as a concise list.
+   - Deduplicate only when the context clearly describes the same event more than once.
 4. For date, time, order, recency, or "when" questions:
-   - Use exact absolute dates from the context whenever available.
-   - Do not answer with relative time words like "yesterday", "today", "last week", or "recently" when an absolute date is available.
+   - If the context uses a relative expression to describe when something happened (e.g. "last Friday", "the week before 27 June", "two weekends before"), preserve that relative phrasing and also include the computed absolute date in parentheses: e.g. "the week before 27 June 2023 (approx. 19–25 June 2023)".
+   - For direct date facts stated as absolute dates, use the absolute date.
    - If the context includes a temporal note, use it to resolve relative wording.
 5. For changed or updated information:
    - Prefer the most recent value when later context overrides earlier context.
    - Mention older values only if the question asks for history or change over time.
+   - When the question asks for "the significant event", "what happened during [period]", or uses a superlative: enumerate all events in that window, then return the most notable one. Milestone events (relationships, major decisions, achievements) take priority over routine events (appointments, errands) unless the question implies otherwise.
 6. Copy critical names, dates, places, titles, quantities, and field names exactly from the context. Do not blur them into a generic paraphrase.
 7. If the context contains any relevant evidence, commit to the most supported answer — even if it requires reasoning. Reserve "I don't have that information" strictly for when the context has zero relevant facts about the subject. Committing to a reasoned answer is preferred over abstaining.
 8. For answers expressed as "N days/weeks/months before/after DATE": use the temporal note in the context to compute the actual calendar date and give it as an absolute date (e.g. "18 May 2023"). Do not echo the anchor date as the answer.
-9. If the context contains facts about multiple named people, only use facts about the person the question asks about. Facts labeled "User" or "Assistant" refer to the primary conversation participant — if other facts establish that person's name (e.g. "User's name is Caroline"), treat "User" facts as belonging to that person.
+9. If the context contains facts about multiple named people, only use facts where the subject explicitly matches the person the question asks about. Before giving your answer, verify: does the fact literally name the person the question is asking about? If a fact says "James did X" and the question asks what John did — that fact is not usable as an answer about John. Do not infer attribution. Facts labeled "User" or "Assistant" refer to the primary conversation participant — if other facts establish that person's name (e.g. "User's name is Caroline"), treat "User" facts as belonging to that person.
+10. Match answer length to question scope. If the question asks for one item, name, date, or fact — give only that. Do not add explanatory sentences, history, or extra context unless the question explicitly asks for reasoning or detail. Extra correct sentences create noise that can obscure the core answer.
 
 ANSWER:
 """