Skip to content

Commit aee23d7

Browse files
monperrusclaude
andcommitted
Normalize double-dash to single dash in title comparison
LaTeX uses '--' for an en-dash; Semantic Scholar stores a plain '-'. Adding .replace("--", "-") before other normalization makes them match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 2dad8f1 commit aee23d7

2 files changed

Lines changed: 7 additions & 0 deletions

File tree

check_hallucinations/main.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ def _normalize_title(title):
3232
title.lower()
3333
.strip()
3434
.rstrip(".")
35+
.replace("--", "-")
3536
.replace("\\$", "$")
3637
.replace("\\%", "%")
3738
.replace("\\&", "&")

test_normalize.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,9 @@ def test_latex_dollar_escape():
2424
bib = r"Fixing 55 out of 105 bugs for \$8 each"
2525
ss = "Fixing 55 out of 105 bugs for $8 each"
2626
assert _normalize_title(bib) == _normalize_title(ss)
27+
28+
29+
def test_double_dash_vs_single_dash():
30+
bib = "DeepSeek-Coder: When the large language model meets programming -- the rise of code intelligence"
31+
ss = "DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence"
32+
assert _normalize_title(bib) == _normalize_title(ss)

0 commit comments

Comments
 (0)