fix(scoring): a Solr operator is not an acronym - #543
Merged
Conversation
The all-caps exception added in #536 so that `UN population` keeps its `UN` also let `OR`, `AND` and `NOT` through: `aria OR acqua` scored on three terms, and a title carrying one of them got 4 × 1/3 instead of 4 × 1/2. Found testing the local MCP client after the merge — the term list in the response read ["aria","or","acqua"]. Solr's boolean keywords are dropped before the acronym rule. Lowercase `or` was already a stopword and stays one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MEpSWpAwuMaGkfnMpQdqK2
|
| Filename | Overview |
|---|---|
| src/tools/package.ts | Updates query-term extraction to distinguish bare Solr operators from quoted literals and correctly skip escaped characters while locating phrase boundaries. |
| tests/unit/package-scoring.test.ts | Adds regression coverage for bare operators, quoted operators, and escaped quotes inside phrases. |
Reviews (3): Last reviewed commit: "fix(scoring): an escaped quote does not ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Review finding on #543: `"OR"` in double quotes is a literal search term to Solr, and dropping every OR token after tokenisation had discarded the quotes left such a query with no terms to score on. It had never scored — before #536 the `or` stopword took it — so this is a step past the regression, not just its repair. Quoted tokens are collected before filtering; an operator is dropped only when it is bare. Test added; 545 tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MEpSWpAwuMaGkfnMpQdqK2
Review finding: the quoted-fragment regex stopped at a backslash-escaped quote, so in `"OR\" AND"` only `OR` counted as quoted and the literal `AND` was dropped as syntax. The regex now skips escaped characters inside the phrase. Test added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MEpSWpAwuMaGkfnMpQdqK2
aborruso
added a commit
that referenced
this pull request
Sep 6, 2026
Ships #542 (migrated-portal notice), #543 (boolean operators are not scored terms), #544 (full-coverage candidate pass and stemming) and #545 (smoke cases for the strict pass). Version bumped in package.json, package-lock.json, manifest.json, server.json (both fields), src/server.ts and src/worker.ts. No tools added or removed, so /health stays at 20. Release gate 16/16, 550 tests. Claude-Session: https://claude.ai/code/session_01MEpSWpAwuMaGkfnMpQdqK2 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found testing the local MCP client after #542:
ckan_find_relevant_datasetsonaria OR acquareported"terms": ["aria","or","acqua"].The all-caps exception added in #536 — so that
UN populationkeeps itsUN— also let Solr's own operators through.ORis all-caps by convention, so it survived the stopword list and became a scoring term: a title carrying one of the two real terms scored 4 × 1/3 = 1.3 instead of 4 × 1/2 = 2. Result counts are unaffected (they come from Solr); the local ranking of every boolean query was diluted.AND,ORandNOTare now dropped before the acronym rule. Lowercaseorwas already a stopword and stays one. Test added; 544 tests; the cross-tool smoke case passes.Verified live on dati.comune.milano.it: terms
["aria","acqua"], title share 2 instead of 1.3.🤖 Generated with Claude Code
https://claude.ai/code/session_01MEpSWpAwuMaGkfnMpQdqK2