Exp/locomo bench - #61

Merged

laxmanclo merged 10 commits into

mainfrom

exp/locomo-bench

May 4, 2026

Commits on May 1, 2026

fix: judge bare-answer guard + generalize date arithmetic; soften instr10 conciseness

Alex-Hunterz
and
claude
committed

Commits on May 3, 2026

feat: judge concurrency + stronger prompt rules for short answers

Alex-Hunterz
and
claude
committed
fix: v4 QA prompt — revert instr3, drop instr5+10; retire flash-lite judge

Alex-Hunterz
and
claude
committed

Commits on May 4, 2026

feat: add answer precision/recall + latency P95 to locomo metrics

Alex-Hunterz
and
claude
committed
fix: extraction anti-paraphrase for user facts + per_session 2→3

Alex-Hunterz
and
claude
committed
fix: raise LoCoMoConfig default top_k 10→15

Alex-Hunterz
and
claude
committed
fix: render episodes before facts in context
Alex-Hunterz
committed
fix: recently/lately temporal note → prior date range
Alex-Hunterz
committed
fix: QA prompt — list scan, anti-paraphrase, temporal, sign inversion
Alex-Hunterz
committed
docs: competitor benchmark table with metrics, F1 explanation
Alex-Hunterz
committed