Skip to content

fix(documents): count documents instead of holdings in org facet#4180

Open
jma wants to merge 1 commit into
rero:stagingfrom
jma:maj-fix-4003
Open

fix(documents): count documents instead of holdings in org facet#4180
jma wants to merge 1 commit into
rero:stagingfrom
jma:maj-fix-4003

Conversation

@jma

@jma jma commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The organisation -> library -> location facet is built on the
nested_holdings nested field, so terms sub-aggregations count holdings,
not documents: counts were inflated whenever a document had several
holdings in the same bucket.

Add a reverse_nested sub-aggregation (record_count) at each level to
count distinct parent documents, and expose it as doc_count in the
serializer. Buckets are also re-sorted by document count, since
Elasticsearch orders terms by holdings count and the displayed number is
now the document count.

Co-Authored-by: Johnny Mariéthoz johnny.mariethoz@rero.ch
Co-Authored-by: Ana Gomes anagoncalves7@icloud.com

@jma
jma marked this pull request as ready for review July 22, 2026 14:00
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The organisation facet aggregation now adds reverse-nested document counts at organisation and location levels. Serializer post-processing rewrites nested buckets to use those counts, removes empty buckets, sorts results, and applies local-view filtering. REST expectations and new unit coverage validate document-based counts, bucket pruning, and ordering.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: document counts in the organisation facet instead of holdings.
Description check ✅ Passed The description matches the code changes and explains the reverse_nested counting and bucket reordering.
Linked Issues check ✅ Passed The PR addresses issue #4003 by fixing facet counts to use distinct document counts rather than holdings counts.
Out of Scope Changes check ✅ Passed The changes stay focused on the document facet counting fix and the related serializer and test updates.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rero_ils/config.py`:
- Around line 2129-2147: Update the nested terms aggregations in the
record-count query so both library and location buckets order by their
reverse-nested record_count in descending document-count order before applying
size limits. Add a regression case with more than 50 libraries and more than 100
locations that verifies the highest-record-count buckets are retained.

In `@rero_ils/modules/documents/serializers/json.py`:
- Around line 178-193: Update the organisation, library, and location bucket
rewriting flow around _rewrite_nested_buckets so returned aggregations no longer
retain stale sum_other_doc_count or doc_count_error_upper_bound metadata from
the nested terms response. Remove those fields or recompute them consistently
for the rewritten buckets, and add coverage using a non-zero metadata case.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7e7f301e-64af-432e-80fb-e1c88fa84bb6

📥 Commits

Reviewing files that changed from the base of the PR and between 32459b2 and 73f7081.

📒 Files selected for processing (4)
  • rero_ils/config.py
  • rero_ils/modules/documents/serializers/json.py
  • tests/api/documents/test_documents_rest.py
  • tests/unit/documents/test_documents_serializers.py

Comment thread rero_ils/config.py
Comment thread rero_ils/modules/documents/serializers/json.py Outdated
@coveralls

coveralls commented Jul 22, 2026

Copy link
Copy Markdown

Coverage Status

Coverage is 91.294%jma:maj-fix-4003 into rero:staging. No base build found for rero:staging.

The organisation -> library -> location facet is built on the
`nested_holdings` nested field, so terms sub-aggregations count holdings,
not documents: counts were inflated whenever a document had several
holdings in the same bucket.

Add a `reverse_nested` sub-aggregation (`record_count`) at each level to
count distinct parent documents, and expose it as `doc_count` in the
serializer.

Order the terms buckets by `record_count` in the query so the `size` cut
keeps the buckets with the most documents: Elasticsearch orders terms by
holdings count by default, so a library or location with many documents
but few holdings could otherwise be dropped before the serializer sees it.
The query now returns the buckets already ordered by document count, so
the serializer no longer re-sorts them.

Drop the `sum_other_doc_count` and `doc_count_error_upper_bound` terms
metadata: they still describe the holdings-count aggregation and no longer
match the rewritten (document-count) buckets. They cannot be recomputed for
documents, as a document may belong to several buckets.

* Closes rero#4003.

Co-Authored-by: Johnny Mariéthoz <johnny.mariethoz@rero.ch>
Co-Authored-by: Ana Gomes <anagoncalves7@icloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The organisation/library facet counts are incorrect

5 participants