Skip to content

refactor: rename LLM alignment score to Judge Agreement - #244

Merged
memadi-nv merged 2 commits into
mainfrom
memadi/update-evaluation-report-terminology
Aug 11, 2026
Merged

refactor: rename LLM alignment score to Judge Agreement#244
memadi-nv merged 2 commits into
mainfrom
memadi/update-evaluation-report-terminology

Conversation

@memadi-nv

Copy link
Copy Markdown
Contributor

Related Issue

This PR will affect the screenshot of the shared report in PR #239

Summary

Replaces the term "LLM alignment score" with "Judge Agreement" across all evaluation-related code, comments, and documentation.

Changes

  • display.py — updated user-facing label, tooltip, and inline comments in the replace-mode judge rendering functions
  • replace_runner.py — updated docstring reference
  • anonymizer.py — updated docstring references in preview() and run()

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring
  • CI, release, or contributor workflow update

Contributor Checklist

  • PR title follows Conventional Commits, for example fix: handle empty entity list
  • Related issue is linked, or a maintainer-owned no-issue reason is documented above
  • For non-trivial changes, a plan document is linked above, or the no-plan reason is documented above
  • Public API impact checked; skills/anonymizer/SKILL.md updated if needed
  • No real PII added to tests, docs, notebooks, fixtures, or artifacts
  • No API keys, service tokens, private keys, credentials, or real endpoint secrets added

Validation

  • Commands run:
  • Skipped checks or known failures:

Documentation and Artifacts

  • Docs updated, or not needed
  • If docs changed: make docs-build passes locally
  • If tutorial sources changed: notebooks regenerated with make convert-notebooks
  • If e2e, benchmark, or model-provider behavior changed: relevant validation is listed above

Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
@memadi-nv
memadi-nv requested a review from a team as a code owner August 11, 2026 18:36
@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR consistently renames the evaluation display term “LLM alignment score” to “Judge Agreement” without changing score computation or evaluation behavior.

  • Updates runtime HTML labels, explanatory text, comments, and API docstrings.
  • Updates display tests to assert the new label.
  • Refreshes generated outputs in three tutorial notebooks.

Confidence Score: 5/5

The PR appears safe to merge because it consistently updates presentation terminology without changing runtime evaluation behavior.

The score calculation, verdict branching, data handling, and public APIs remain unchanged, while source labels, tests, docstrings, and generated notebook outputs use the new terminology consistently.

Important Files Changed

Filename Overview
src/anonymizer/interface/display.py Renames the rendered score label and clarifies the detection-judge explanation without altering verdict or score calculations.
src/anonymizer/interface/anonymizer.py Updates run and preview docstrings to use the new evaluation terminology.
src/anonymizer/engine/replace/replace_runner.py Updates the replacement workflow docstring to use the new score name.
tests/interface/test_display.py Strengthens the existing badge test to assert the exact new label and score formatting.
docs/notebooks/01_your_first_anonymization.ipynb Refreshes saved evaluation output to show the new terminology and explanation.
docs/notebooks/02_inspecting_detected_entities.ipynb Refreshes saved detection-evaluation output to show the new terminology.
docs/notebooks/03_choosing_a_replacement_strategy.ipynb Refreshes saved replacement-strategy evaluation outputs to show the new terminology consistently.

Reviews (1): Last reviewed commit: "nit" | Re-trigger Greptile

@memadi-nv
memadi-nv merged commit 7403883 into main Aug 11, 2026
16 checks passed
@memadi-nv
memadi-nv deleted the memadi/update-evaluation-report-terminology branch August 11, 2026 19:25
memadi-nv added a commit that referenced this pull request Aug 13, 2026
* update LLM alignment score naming

Signed-off-by: memadi <memadi@nvidia.com>

* nit

Signed-off-by: memadi <memadi@nvidia.com>

---------

Signed-off-by: memadi <memadi@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants