Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion docs/concepts/choosing-a-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,12 @@ What to include:
- The domain (clinical, legal, financial, customer support, etc.)
- The genre (notes, transcripts, opinions, biographies)
- Anything about the source the engine couldn't infer from a single record (e.g. "transcribed phone calls — expect disfluencies")
- `data_summary` is the only way to provide a soft do-not-tag list for the augmenter when `entity_labels=None` — the augmenter is free to invent labels beyond `DEFAULT_ENTITY_LABELS`, so use it to tell the LLM what *not* to tag (e.g. "do not tag generic anatomical terms, medication class names, or job titles as PII").
- `data_summary` is a soft way to guide the augmenter when `entity_labels=None` — the augmenter is free to invent labels beyond `DEFAULT_ENTITY_LABELS`, so use it to tell the LLM what *not* to tag (e.g. "do not tag generic anatomical terms, medication class names, or job titles as PII"). For a hard exclusion of specific label types, use `Detect.entity_label_denylist` instead.

What to leave out:

- Lists of entity types **you want detected** (those go in `Detect.entity_labels`)
- Lists of entity types **you never want detected** (those go in `Detect.entity_label_denylist`)
- Privacy/utility goals (those go in `Rewrite.privacy_goal`)
- Substitute behavior instructions (e.g. "names should remain Portuguese", "preserve numeric magnitude") — those go in `Substitute(instructions)`
- Generic phrasing ("text data" adds no signal)
Expand Down Expand Up @@ -78,6 +79,18 @@ from anonymizer import DEFAULT_ENTITY_LABELS, Detect
detect = Detect(entity_labels=[*DEFAULT_ENTITY_LABELS, "clinical_facility", "diagnosis_code", "medication_name"])
```

### `entity_label_denylist`

Use when you want to **exclude** specific label types from detection without enumerating the entire allowlist. Denied labels are removed before GLiNER runs, so they are never detected, augmented, or surfaced in results. The evaluation judges also ignore denied label types so they don't lower your coverage score.

```python
# Never detect occupation or gender, keep everything else
Detect(entity_label_denylist=["occupation", "gender"])

# Combine with an explicit allowlist — denylist always wins
Detect(entity_labels=["first_name", "email", "city"], entity_label_denylist=["city"])
```

### `gliner_threshold`

Default `0.3`. The validator catches false positives downstream, so erring low is safe.
Expand Down
16 changes: 16 additions & 0 deletions docs/concepts/detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ config = AnonymizerConfig(
| Field | Default | Description |
|-------|---------|-------------|
| `entity_labels` | `None` (all defaults) | List of labels to detect. Leave unset (or pass `None`) to use the full default set. |
| `entity_label_denylist` | `None` | List of labels to **never** detect, even if present in `entity_labels` or the default set. Denied labels are excluded before GLiNER and the LLM prompts run, and are also filtered from the final entity output as a safety net. |
| `gliner_threshold` | `0.3` | GLiNER confidence threshold (0.0--1.0). Lower values detect more entities but may increase false positives. |
| `validation_max_entities_per_call` | `100` | Maximum candidate entities per validator LLM call. Rows with more candidates are split into chunks. See [Chunked validation](#chunked-validation). |
| `validation_excerpt_window_chars` | `500` | Characters of context included before and after a chunk's entity spans in the validator prompt. Bounds per-chunk prompt size; not the model's context-window limit. |
Expand Down Expand Up @@ -104,6 +105,21 @@ Detect(entity_labels=["first_name", "last_name", "email"])
# Permissive: detect all defaults + LLM can infer new label types
Detect() # entity_labels=None
```

### Excluding labels with a deny list

Use `entity_label_denylist` to exclude specific labels from detection without having to enumerate the entire allowlist. Denied labels are removed before GLiNER runs and before the LLM prompts are built, so they are never detected or augmented.

```python
# Detect all defaults except occupation and gender
Detect(entity_label_denylist=["occupation", "gender"])

# Combine with an explicit allowlist — denylist always wins
Detect(entity_labels=["first_name", "email", "city"], entity_label_denylist=["city"])
```

!!! warning
If every label in `entity_labels` is also in `entity_label_denylist`, the effective detection set is empty and no entities will be detected. Anonymizer logs a warning when this happens.
## Tuning the threshold

For `gliner_threshold`, start with the default `0.3`. If you're seeing too many false positives, raise it to `0.5`. If entities are being missed, try lowering to `0.2`. The LLM validation step catches many false positives, so erring on the side of lower thresholds is usually safe.
Expand Down
1 change: 1 addition & 0 deletions docs/concepts/evaluation.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ Note: the judge measures detection recall, not output leakage. A value detected
The judge is scoped and contextualized by the same signals used during anonymization:

- **`entity_labels`** — the detection taxonomy in scope; the judge only reports values whose type falls within it.
- **`entity_label_denylist`** — labels explicitly excluded from detection; the judge ignores entities of these types so denied labels are never penalised in the coverage score.
- **`data_summary`** — used purely to interpret literal values and their semantic types, never to invent entities absent from the text.

| Output column | Type | Description |
Expand Down
10 changes: 7 additions & 3 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,13 @@ Verify by re-running `preview` with `Annotate` and confirming the entity now app

Symptoms: detected entities include obvious common words, dates that aren't dates, etc.

1. **Raise `gliner_threshold`** to `0.5`. The augmenter will pick up real misses, so this rarely costs recall.
2. **Lower `validation_excerpt_window_chars`** (default `500`) if context-driven validation is being misled by far-away sentences. Smaller per-chunk prompts trade context for precision.
3. **Sanity-check the validator with an `Annotate` preview.** A flaky validator (or a misconfigured alias) returns "keep" on almost everything, which presents as recall going way up — easiest spotted by eyeballing the entity list on a handful of rows.
1. **Use `Detect.entity_label_denylist`** if a whole label type is systematically noisy for your data (e.g. `occupation` tagging generic job words, `age` tagging durations). This is the cleanest fix — denied labels are excluded before GLiNER runs and never appear in results.
```python
Detect(entity_label_denylist=["occupation", "age"])
```
2. **Raise `gliner_threshold`** to `0.5`. The augmenter will pick up real misses, so this rarely costs recall.
3. **Lower `validation_excerpt_window_chars`** (default `500`) if context-driven validation is being misled by far-away sentences. Smaller per-chunk prompts trade context for precision.
4. **Sanity-check the validator with an `Annotate` preview.** A flaky validator (or a misconfigured alias) returns "keep" on almost everything, which presents as recall going way up — easiest spotted by eyeballing the entity list on a handful of rows.

### A new domain isn't being detected well

Expand Down
1 change: 1 addition & 0 deletions skills/anonymizer/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ regulatory and business context.
# Usage Tips and Common Pitfalls

- **`Detect.entity_labels=None` (the default) is permissive** — the augmenter LLM may invent labels not in `DEFAULT_ENTITY_LABELS`. Setting an explicit list switches to **strict mode** where *only* the listed labels are detected. To add domain labels, *extend* the default, don't replace it: `entity_labels=[*DEFAULT_ENTITY_LABELS, "clinical_facility", ...]` (`DEFAULT_ENTITY_LABELS` is a tuple, so unpack it into a list). Match the snake_case convention of `DEFAULT_ENTITY_LABELS`.
- **`Detect.entity_label_denylist`** excludes specific label types from detection entirely — denied labels are removed before GLiNER runs and are never detected, augmented, or penalised in evaluation scores. Use it when a label type is systematically noisy for your data or should never be anonymized (e.g. `Detect(entity_label_denylist=["occupation", "gender"])`). The denylist takes precedence over `entity_labels` — a label in both is never detected.
- **GLiNER is zero-shot** — entity labels are natural-language concept names (e.g. `"clinical_facility"`, `"internal_project_codename"`), not codes or enum values. Any concept you can name in English is a label GLiNER can detect.
- **`Rewrite.instructions` is a dead field today** — it exists on the model but the rewrite engine never reads it. Do not use it. Put rewriter guidance in `privacy_goal.protect` / `privacy_goal.preserve` instead.
- **`risk_tolerance` only applies to Rewrite mode**, not Replace.
Expand Down
32 changes: 32 additions & 0 deletions src/anonymizer/config/anonymizer_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,14 @@ class Detect(BaseModel):
"To inspect the default set, use `from anonymizer import DEFAULT_ENTITY_LABELS`."
),
)
entity_label_denylist: list[str] | None = Field(
default=None,
description=(
"Entity labels to never detect, even if present in entity_labels or the default set. "
"Denied labels are excluded before GLiNER and LLM prompts run, and are also filtered "
"from the final entity output as a safety net."
),
)
gliner_threshold: float = Field(
default=0.3, ge=0.0, le=1.0, description="GLiNER detection confidence threshold (0.0-1.0)."
)
Expand Down Expand Up @@ -114,6 +122,30 @@ def validate_entity_labels(cls, value: list[str] | None) -> list[str] | None:
logger.warning("entity_labels contained duplicates, removed automatically.")
return deduped

@field_validator("entity_label_denylist")
@classmethod
def validate_entity_label_denylist(cls, value: list[str] | None) -> list[str] | None:
if value is None:
return value
cleaned = [label.strip().lower() for label in value if label.strip()]
if not cleaned:
raise ValueError("entity_label_denylist must not be empty. Use None to disable the deny list.")
deduped = sorted(set(cleaned))
if len(deduped) != len(cleaned):
logger.warning("entity_label_denylist contained duplicates, removed automatically.")
return deduped

@model_validator(mode="after")
def warn_on_allowlist_denylist_overlap(self) -> "Detect":
if self.entity_labels is not None and self.entity_label_denylist is not None:
overlap = sorted(set(self.entity_labels) & set(self.entity_label_denylist))
if overlap:
logger.warning(
"entity_labels and entity_label_denylist share labels that will never be detected: %s",
overlap,
)
return self


class Rewrite(BaseModel):
"""Configuration for rewrite-mode execution."""
Expand Down
Loading
Loading