Task Summary
The problem: we use yaml to expose user-controllable knobs, including the new PII replacement plan that we want users to audit. However, yaml configs often appear complicated, especially if it includes lists/dictionaries and understanding the yaml syntax is a prerequisite to understanding the configs.
The solution: inspired by the Anonymizer preview, we will build a replacement plan preview that renders in notebooks, in the shape of a table, where we highlight the associated columns, columns to replace and entity types etc.
Technical Details & Implementation Plan
The same pattern Anonymizer uses works well here, and tabular PII is arguably a better fit because the "preview" is mostly metadata about columns, not inline text spans.
What you already have
Safe-Synthesizer's plan model maps cleanly to the visuals you want:
| Concept |
Source |
| Related column groups |
PiiReplacementPlan.associated_column_sets (keyed by role, e.g. patient, doctor) |
| Standalone columns |
unassociated_columns_to_replace |
| Entity type per column |
PiiColumnPlan.entity_type |
| Grouping key |
plan.group_key |
| "Full column" vs "scan inside cells" |
entity_type != free_text → whole-column structured replace; free_text → NER pass per cell |
Discovery already produces this via resolve_plan() / discover_plan() without needing to run replacement. Today that only happens inside TabularPiiReplacer.transform_df(), but you could expose a preview_plan(df) that stops after resolve_plan().
Recommended approach (mirror Anonymizer)
Use the same stack:
- Build HTML in a
display.py-style module (hand-rolled HTML + html.escape, like Anonymizer).
- Render in notebooks via
IPython.display.HTML + display().
- CLI fallback — print a plain-text summary or write HTML to a file.
I'd split it into two panels rather than one giant styled DataFrame:
1. Plan manifest (column metadata)
A table with one row per affected column:
- Column name
- Entity badge (
first_name, email, free_text, …)
- Role group (
patient, doctor, or "unassociated")
- Replacement mode: full column vs partial (NER in cells)
- Conditioning columns (if any)
- Optional: sample values / cardinality from detection
This is the clearest place for entity tags and "will be completely replaced" semantics.
2. Sample data with grouped column headers
Show df.head(n) with original values (no synthetic output). Style column headers, not cells:
- Color band per
associated_column_sets role (reuse Anonymizer's hash-to-color pattern)
- Unassociated columns get a neutral style
- Untouched columns can be dimmed or omitted
group_key gets a distinct marker
HTML <colgroup> / multi-row <thead> with colspan works better than pandas Styler for role bands across wide tables.
Dependencies
No response
Task Summary
The problem: we use yaml to expose user-controllable knobs, including the new PII replacement plan that we want users to audit. However, yaml configs often appear complicated, especially if it includes lists/dictionaries and understanding the yaml syntax is a prerequisite to understanding the configs.
The solution: inspired by the Anonymizer preview, we will build a replacement plan preview that renders in notebooks, in the shape of a table, where we highlight the associated columns, columns to replace and entity types etc.
Technical Details & Implementation Plan
The same pattern Anonymizer uses works well here, and tabular PII is arguably a better fit because the "preview" is mostly metadata about columns, not inline text spans.
What you already have
Safe-Synthesizer's plan model maps cleanly to the visuals you want:
PiiReplacementPlan.associated_column_sets(keyed by role, e.g.patient,doctor)unassociated_columns_to_replacePiiColumnPlan.entity_typeplan.group_keyentity_type != free_text→ whole-column structured replace;free_text→ NER pass per cellDiscovery already produces this via
resolve_plan()/discover_plan()without needing to run replacement. Today that only happens insideTabularPiiReplacer.transform_df(), but you could expose apreview_plan(df)that stops afterresolve_plan().Recommended approach (mirror Anonymizer)
Use the same stack:
display.py-style module (hand-rolled HTML +html.escape, like Anonymizer).IPython.display.HTML+display().I'd split it into two panels rather than one giant styled DataFrame:
1. Plan manifest (column metadata)
A table with one row per affected column:
first_name,email,free_text, …)patient,doctor, or "unassociated")This is the clearest place for entity tags and "will be completely replaced" semantics.
2. Sample data with grouped column headers
Show
df.head(n)with original values (no synthetic output). Style column headers, not cells:associated_column_setsrole (reuse Anonymizer's hash-to-color pattern)group_keygets a distinct markerHTML
<colgroup>/ multi-row<thead>withcolspanworks better than pandasStylerfor role bands across wide tables.Dependencies
No response