Skip to content

Commit b667b0f

Browse files
committed
docs: clarify rewrite notebook guidance
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
1 parent bf934f7 commit b667b0f

7 files changed

Lines changed: 33 additions & 16 deletions

docs/notebook_source/01_your_first_anonymization.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
14
# ---
25
# jupyter:
36
# jupytext:

docs/notebook_source/02_inspecting_detected_entities.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
14
# ---
25
# jupyter:
36
# jupytext:

docs/notebook_source/03_choosing_a_replacement_strategy.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
14
# ---
25
# jupyter:
36
# jupytext:

docs/notebook_source/04_rewriting_biographies.py

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
14
# ---
25
# jupyter:
36
# jupytext:
@@ -18,8 +21,8 @@
1821
# -->
1922
# # 🕵️ Rewriting Biographies
2023
#
21-
# Instead of replacing entities with tokens, rewrite mode generates a
22-
# privacy-safe transformation of the entire text. The `run()` / `preview()` pipeline:
24+
# Instead of replacing individual entities, rewrite mode transforms the entire
25+
# text and measures any sensitive information that remains. The `run()` / `preview()` pipeline:
2326
#
2427
# 1. Detects entities (same as replace mode, plus latent entity detection)
2528
# 2. Classifies the domain and assigns sensitivity dispositions
@@ -154,7 +157,7 @@
154157
# %% [markdown]
155158
# ## 🚩 Filter by review flag
156159
#
157-
# - Records where automated metrics exceed thresholds are flagged for manual review.
160+
# - Records that cross the configured leakage or utility thresholds are flagged for manual review.
158161
# - `needs_human_review` is threshold-based, so a record can have small nonzero
159162
# leakage without being flagged.
160163
# - Use this to prioritize human attention on the records that need it most.
@@ -173,7 +176,7 @@
173176
# Call `evaluate()` to run LLM-as-judge scoring on the rewrite result — detection validity and three quality rubrics (privacy, quality, style).
174177
# Evaluation makes additional LLM calls per record. For larger datasets, evaluate
175178
# a preview first; this tutorial evaluates all 25 rows to demonstrate the complete workflow.
176-
# This holistic judge is independent of pipeline leakage scoring, so their assessments may differ.
179+
# The holistic privacy rubric and pipeline leakage metric are independent, so they may disagree.
177180
# See [Evaluation](../../concepts/evaluation/#rewrite-evaluation) for details.
178181

179182
# %%

docs/notebook_source/05_rewriting_legal_documents.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
14
# ---
25
# jupyter:
36
# jupytext:
@@ -178,9 +181,10 @@
178181
# %% [markdown]
179182
# ## 🚩 Filter by review flag
180183
#
181-
# - Records where automated metrics exceed thresholds are flagged for manual review.
182-
# - The repair loop stops after `max_repair_iterations`; records that still need
183-
# repair remain flagged for human review but are not pipeline failures.
184+
# - Records that cross the configured leakage or utility thresholds are flagged for manual review.
185+
# - The repair loop stops after `max_repair_iterations`. Afterward,
186+
# `needs_human_review` is computed separately from the final leakage, utility,
187+
# and high-sensitivity-leak metrics.
184188
# - Use this to prioritize human attention on the records that need it most.
185189
# - See [Working with flagged records](../../concepts/rewrite/#working-with-flagged-records)
186190
# for guidance on diagnosing and resolving flagged records.
@@ -197,7 +201,7 @@
197201
# Call `evaluate()` to run LLM-as-judge scoring on the rewrite result — detection validity and three quality rubrics (privacy, quality, style).
198202
# Evaluation makes additional LLM calls per record. For larger datasets, evaluate
199203
# a preview first; this tutorial evaluates all 25 rows to demonstrate the complete workflow.
200-
# This holistic judge is independent of pipeline leakage scoring, so their assessments may differ.
204+
# The holistic privacy rubric and pipeline leakage metric are independent, so they may disagree.
201205
# See [Evaluation](../../concepts/evaluation/#rewrite-evaluation) for details.
202206

203207
# %%

docs/notebooks/04_rewriting_biographies.ipynb

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@
1111
"-->\n",
1212
"# 🕵️ Rewriting Biographies\n",
1313
"\n",
14-
"Instead of replacing entities with tokens, rewrite mode generates a\n",
15-
"privacy-safe transformation of the entire text. The `run()` / `preview()` pipeline:\n",
14+
"Instead of replacing individual entities, rewrite mode transforms the entire\n",
15+
"text and measures any sensitive information that remains. The `run()` / `preview()` pipeline:\n",
1616
"\n",
1717
"1. Detects entities (same as replace mode, plus latent entity detection)\n",
1818
"2. Classifies the domain and assigns sensitivity dispositions\n",
@@ -800,7 +800,7 @@
800800
"source": [
801801
"## 🚩 Filter by review flag\n",
802802
"\n",
803-
"- Records where automated metrics exceed thresholds are flagged for manual review.\n",
803+
"- Records that cross the configured leakage or utility thresholds are flagged for manual review.\n",
804804
"- `needs_human_review` is threshold-based, so a record can have small nonzero\n",
805805
" leakage without being flagged.\n",
806806
"- Use this to prioritize human attention on the records that need it most.\n",
@@ -891,7 +891,7 @@
891891
"Call `evaluate()` to run LLM-as-judge scoring on the rewrite result — detection validity and three quality rubrics (privacy, quality, style).\n",
892892
"Evaluation makes additional LLM calls per record. For larger datasets, evaluate\n",
893893
"a preview first; this tutorial evaluates all 25 rows to demonstrate the complete workflow.\n",
894-
"This holistic judge is independent of pipeline leakage scoring, so their assessments may differ.\n",
894+
"The holistic privacy rubric and pipeline leakage metric are independent, so they may disagree.\n",
895895
"See [Evaluation](../../concepts/evaluation/#rewrite-evaluation) for details."
896896
]
897897
},

docs/notebooks/05_rewriting_legal_documents.ipynb

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -960,9 +960,10 @@
960960
"source": [
961961
"## 🚩 Filter by review flag\n",
962962
"\n",
963-
"- Records where automated metrics exceed thresholds are flagged for manual review.\n",
964-
"- The repair loop stops after `max_repair_iterations`; records that still need\n",
965-
" repair remain flagged for human review but are not pipeline failures.\n",
963+
"- Records that cross the configured leakage or utility thresholds are flagged for manual review.\n",
964+
"- The repair loop stops after `max_repair_iterations`. Afterward,\n",
965+
" `needs_human_review` is computed separately from the final leakage, utility,\n",
966+
" and high-sensitivity-leak metrics.\n",
966967
"- Use this to prioritize human attention on the records that need it most.\n",
967968
"- See [Working with flagged records](../../concepts/rewrite/#working-with-flagged-records)\n",
968969
" for guidance on diagnosing and resolving flagged records."
@@ -1136,7 +1137,7 @@
11361137
"Call `evaluate()` to run LLM-as-judge scoring on the rewrite result — detection validity and three quality rubrics (privacy, quality, style).\n",
11371138
"Evaluation makes additional LLM calls per record. For larger datasets, evaluate\n",
11381139
"a preview first; this tutorial evaluates all 25 rows to demonstrate the complete workflow.\n",
1139-
"This holistic judge is independent of pipeline leakage scoring, so their assessments may differ.\n",
1140+
"The holistic privacy rubric and pipeline leakage metric are independent, so they may disagree.\n",
11401141
"See [Evaluation](../../concepts/evaluation/#rewrite-evaluation) for details."
11411142
]
11421143
},

0 commit comments

Comments
 (0)