Add PBI Fixer UI: interactive ipywidgets app for Power BI development#1162
Add PBI Fixer UI: interactive ipywidgets app for Power BI development#1162KornAlexander wants to merge 1 commit intomicrosoft:mainfrom
Conversation
…r BI reports and semantic models
There was a problem hiding this comment.
Pull request overview
Adds the interactive “explorer/editor” tabs that power the PBI Fixer notebook UI, providing tree-based navigation and inline editing for both PBIR reports and semantic models.
Changes:
- Introduces a Report Explorer tab to browse pages/visuals, run fixers, and edit basic layout properties.
- Introduces a Model Explorer tab to browse semantic model objects, preview/edit expressions, run fixers, and refresh model/table/partition.
- Introduces a Perspective Editor tab to manage model perspectives with a tri-state checkbox tree.
Reviewed changes
Copilot reviewed 2 out of 4 changed files in this pull request and generated 13 comments.
| File | Description |
|---|---|
src/sempy_labs/_report_explorer.py |
New ipywidgets Report Explorer tab (tree, scan/run fixers, preview, property editing). |
src/sempy_labs/_model_explorer.py |
New ipywidgets Model Explorer tab (tree, expression/table preview, fixers, refresh, scan results). |
src/sempy_labs/_perspective_editor.py |
New ipywidgets Perspective Editor tab (tri-state tree and CRUD for perspectives). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| v_raw = key.split(":", 1)[1] | ||
| v_parts = v_raw.rsplit("\x1f", 1) | ||
| v_name = v_parts[-1] | ||
| # Find page for this visual | ||
| p_name = None | ||
| pages_df = rw.list_pages() | ||
| visuals_df = rw.list_visuals() | ||
| for _, vr in visuals_df.iterrows(): | ||
| if str(vr.get("Visual Name", "")) == v_name: | ||
| p_name = str(vr.get("Page Name", "")) |
There was a problem hiding this comment.
on_rp_save() parses visual keys incorrectly: v_name becomes the entire "{page}:{visual}" string (and in multi-report also includes the page prefix), so the subsequent list_visuals() lookup will never match and visual property changes won’t be saved. Parse visual keys using key.split(':', 2) (page part in parts[1], visual name in parts[2]) and use both page + visual to locate the correct visual row.
| v_raw = key.split(":", 1)[1] | |
| v_parts = v_raw.rsplit("\x1f", 1) | |
| v_name = v_parts[-1] | |
| # Find page for this visual | |
| p_name = None | |
| pages_df = rw.list_pages() | |
| visuals_df = rw.list_visuals() | |
| for _, vr in visuals_df.iterrows(): | |
| if str(vr.get("Visual Name", "")) == v_name: | |
| p_name = str(vr.get("Page Name", "")) | |
| parts = key.split(":", 2) | |
| p_name = None | |
| v_name = None | |
| if len(parts) == 3: | |
| p_name = parts[1] | |
| v_name = parts[2] | |
| if "\x1f" in p_name: | |
| _, p_name = p_name.split("\x1f", 1) | |
| if "\x1f" in v_name: | |
| _, v_name = v_name.split("\x1f", 1) | |
| visuals_df = rw.list_visuals() | |
| for _, vr in visuals_df.iterrows(): | |
| if ( | |
| str(vr.get("Page Name", "")) == p_name | |
| and str(vr.get("Visual Name", "")) == v_name | |
| ): | |
| p_name = str(vr.get("Page Name", "")) | |
| v_name = str(vr.get("Visual Name", "")) |
| finally: | ||
| sys.stdout = _old | ||
| _rp_mark_clean() | ||
| set_status(save_status, f"✓ Saved {len(_rp_pending)} change(s).", "#34c759") | ||
| except Exception as e: |
There was a problem hiding this comment.
In _save_bg(), _rp_mark_clean() clears _rp_pending before the success message is built, so the UI will always report “Saved 0 change(s)”. Capture the pending change count before calling _rp_mark_clean() (or have _rp_mark_clean() return the count) and use that for the status message.
| rpt = rpt_input.split(",")[0].strip() | ||
| for pfx in ("\U0001F4C4 ", "\U0001F4CA "): | ||
| if rpt.startswith(pfx): | ||
| rpt = rpt[len(pfx):] |
There was a problem hiding this comment.
Saving report edits currently derives rpt from the first comma-separated value in report_input, but _rp_pending keys can include multiple reports (and even when report_input is blank, multi-report load is supported). This means edits for non-first reports can’t be saved (or saving may fail entirely). Consider either disabling editing/saving in multi-report mode or grouping _rp_pending by report and opening connect_report() per report.
| rpt = rpt_input.split(",")[0].strip() | |
| for pfx in ("\U0001F4C4 ", "\U0001F4CA "): | |
| if rpt.startswith(pfx): | |
| rpt = rpt[len(pfx):] | |
| def _normalize_report_name(name): | |
| name = (name or "").strip() | |
| for pfx in ("\U0001F4C4 ", "\U0001F4CA "): | |
| if name.startswith(pfx): | |
| name = name[len(pfx):] | |
| return name.strip() | |
| report_names = [ | |
| _normalize_report_name(part) | |
| for part in rpt_input.split(",") | |
| if _normalize_report_name(part) | |
| ] | |
| pending_reports = { | |
| _normalize_report_name(key.split("\x1f", 1)[0]) | |
| for key in _rp_pending | |
| if "\x1f" in key and _normalize_report_name(key.split("\x1f", 1)[0]) | |
| } | |
| if pending_reports: | |
| if len(pending_reports) > 1: | |
| set_status( | |
| save_status, | |
| "Saving edits is not supported when pending changes span multiple reports.", | |
| "#ff3b30", | |
| ) | |
| return | |
| rpt = next(iter(pending_reports)) | |
| elif len(report_names) == 1: | |
| rpt = report_names[0] | |
| elif len(report_names) > 1: | |
| set_status( | |
| save_status, | |
| "Saving edits is not supported in multi-report mode.", | |
| "#ff3b30", | |
| ) | |
| return | |
| else: | |
| rpt = "" |
| if node_type == "page": | ||
| p_name = key.split(":", 1)[1] | ||
| if "\x1f" in p_name: | ||
| _, p_name = p_name.split("\x1f", 1) | ||
| pages = _report_data.get("pages", {}) | ||
| if not pages and _report_data.get("reports"): | ||
| for rd in _report_data["reports"].values(): | ||
| pages.update(rd.get("pages", {})) | ||
| p = pages.get(p_name, {}) |
There was a problem hiding this comment.
_populate_report_props() collapses multi-report context by merging pages from all reports into a single pages dict keyed only by page name. If two reports share a page internal name, property display/editing can target the wrong page/visual. Use the report-qualified key (e.g., via _resolve_page() with the full p_key) instead of merging pages across reports.
| badge = f'<span style="color:#888;">{fmt or "unknown"}</span>' | ||
| html += f'<tr><td style="padding:2px 12px 2px 8px; border-bottom:1px solid #f0f0f0; white-space:nowrap;">{name}</td>' | ||
| html += f'<td style="padding:2px 8px; border-bottom:1px solid #f0f0f0; white-space:nowrap;">{badge}</td></tr>' | ||
| html += '</table>' |
There was a problem hiding this comment.
Several HTML panels interpolate report names and other metadata directly into widgets.HTML (e.g., the PBIR Status table). Report names can contain characters like </&, which will be rendered as HTML and can lead to injection in the notebook UI. Escape dynamic text with html.escape() before embedding into HTML strings.
| html += '<tr style="background:#f5f5f5; position:sticky; top:0;">' | ||
| for col in df.columns: | ||
| html += f'<th style="padding:3px 6px; border-bottom:2px solid #e0e0e0; text-align:left; white-space:nowrap;">{col}</th>' | ||
| html += '</tr>' | ||
| for _, row in df.iterrows(): | ||
| html += '<tr>' | ||
| for col in df.columns: | ||
| val = row[col] | ||
| if val is None or (isinstance(val, float) and val != val): | ||
| val = "" | ||
| html += f'<td style="padding:2px 6px; border-bottom:1px solid #f0f0f0; white-space:nowrap;">{val}</td>' | ||
| html += '</tr>' |
There was a problem hiding this comment.
_load_table_preview() renders cell values directly into an HTML table without escaping. String values from the model can include </& and will be interpreted as HTML in widgets.HTML, creating an injection vector. Escape headers and cell values with html.escape(str(val)) before concatenating.
| resp = requests.post("https://daxformatter.azurewebsites.net/api/daxformatter/daxtextformatmulti", json=payload, headers=headers) | ||
| result = resp.json() | ||
| if result and result[0].get("formatted"): | ||
| txt = result[0]["formatted"] | ||
| if txt.startswith("x :="): | ||
| txt = txt[4:] | ||
| if txt.startswith("\r\n"): | ||
| txt = txt[2:] | ||
| elif txt.startswith("\n"): | ||
| txt = txt[1:] | ||
| old_val = preview.value | ||
| _suppressing_observe[0] = True | ||
| preview.value = txt | ||
| _suppressing_observe[0] = False | ||
| # Mark as dirty if formatting actually changed the text | ||
| if txt != old_val: | ||
| _capture_current() | ||
| _tree_stale[0] = True | ||
| n = len(_pending_changes) | ||
| save_btn.description = f"\u26a0\ufe0f {n} unsaved change(s)" | ||
| save_btn.button_style = "danger" | ||
| save_btn.disabled = False | ||
| discard_btn.layout.display = "" | ||
| except Exception: | ||
| pass | ||
| btn.disabled = False | ||
| btn.description = orig |
There was a problem hiding this comment.
The DAX formatter call uses requests.post(...) without a timeout and swallows all exceptions, which can leave the UI stuck waiting on a network call with no feedback. Add a reasonable timeout= and surface failures via set_status() (or at least restore the button label and show an error).
| f'<tr style="background:{bg};">' | ||
| f'<td style="padding:4px 6px;font-size:11px;white-space:nowrap;vertical-align:top;">{severity_icon}</td>' | ||
| f'<td style="padding:4px 6px;font-size:11px;white-space:nowrap;vertical-align:top;color:#333;">{ds}</td>' | ||
| f'<td style="padding:4px 6px;font-size:11px;white-space:nowrap;vertical-align:top;color:{ICON_ACCENT};font-weight:600;">{fixer_name.strip()}</td>' | ||
| f'<td style="padding:4px 6px;font-size:11px;vertical-align:top;color:#555;word-break:break-word;">{detail}</td>' | ||
| f'</tr>' |
There was a problem hiding this comment.
Scan results HTML interpolates detail directly into a <td> without escaping. Since detail comes from fixer stdout, it can contain </& and be rendered as HTML in the notebook UI. Escape ds, fixer_name, and detail before embedding into the HTML table.
| # Tables and objects | ||
| for table in tm.model.Tables: | ||
| t_name = table.Name | ||
| if table.IsHidden: | ||
| data["hidden_tables"].add(t_name) | ||
| columns = sorted(c.Name for c in tm.all_columns() if c.Parent == table) | ||
| measures = sorted(m.Name for m in table.Measures) | ||
| hierarchies = sorted(h.Name for h in table.Hierarchies) | ||
| for c in tm.all_columns(): | ||
| if c.Parent == table and c.IsHidden: |
There was a problem hiding this comment.
_load_perspective_data() iterates tm.all_columns() multiple times per table (once to build columns, and again to find hidden columns). Since all_columns() loops all tables, this becomes O(tables × columns) and can be very slow on large models. Materialize all_columns = list(tm.all_columns()) once and group/filter by parent table.
| # Tables and objects | |
| for table in tm.model.Tables: | |
| t_name = table.Name | |
| if table.IsHidden: | |
| data["hidden_tables"].add(t_name) | |
| columns = sorted(c.Name for c in tm.all_columns() if c.Parent == table) | |
| measures = sorted(m.Name for m in table.Measures) | |
| hierarchies = sorted(h.Name for h in table.Hierarchies) | |
| for c in tm.all_columns(): | |
| if c.Parent == table and c.IsHidden: | |
| all_columns = list(tm.all_columns()) | |
| columns_by_table = {} | |
| for column in all_columns: | |
| columns_by_table.setdefault(column.Parent, []).append(column) | |
| # Tables and objects | |
| for table in tm.model.Tables: | |
| t_name = table.Name | |
| if table.IsHidden: | |
| data["hidden_tables"].add(t_name) | |
| table_columns = columns_by_table.get(table, []) | |
| columns = sorted(c.Name for c in table_columns) | |
| measures = sorted(m.Name for m in table.Measures) | |
| hierarchies = sorted(h.Name for h in table.Hierarchies) | |
| for c in table_columns: | |
| if c.IsHidden: |
| members = _data.get("perspective_members", {}).get(persp_name, {}) | ||
| for table_name in _data.get("tables", {}): | ||
| tbl_members = members.get(table_name, {}) | ||
| c_list = _child_cbs_map.get(table_name, []) | ||
| for cb, obj_type, obj_name in c_list: | ||
| cb.value = obj_name in tbl_members.get(obj_type, set()) | ||
| _loading[0] = False | ||
| # Manually sync all UI state (observers were suppressed) | ||
| for table_name in _data.get("tables", {}): | ||
| c_list = _child_cbs_map.get(table_name, []) | ||
| vals = [cb.value for cb, _, _ in c_list] | ||
| # Table checkbox | ||
| if table_name in _table_cbs: | ||
| _table_cbs[table_name].value = bool(any(vals)) | ||
| # Status icon | ||
| if table_name in _status_icons: | ||
| if all(vals) and vals: | ||
| _status_icons[table_name].value = ICON_ALL | ||
| elif any(vals): | ||
| _status_icons[table_name].value = ICON_SOME | ||
| else: | ||
| _status_icons[table_name].value = ICON_NONE | ||
| # Table summary | ||
| if table_name in _table_summary_labels: | ||
| tbl_data = _data["tables"][table_name] | ||
| totals = {k: len(v) for k, v in tbl_data.items()} | ||
| counts = {"columns": 0, "measures": 0, "hierarchies": 0} | ||
| for cb, obj_type, _ in c_list: | ||
| if cb.value: | ||
| counts[obj_type] += 1 | ||
| _table_summary_labels[table_name].value = ( | ||
| f'<span style="font-size:12px; color:{table_summary_color}; margin-left:8px;">' | ||
| f'{counts["columns"]}/{totals["columns"]} columns, ' | ||
| f'{counts["measures"]}/{totals["measures"]} measures, ' | ||
| f'{counts["hierarchies"]}/{totals["hierarchies"]} hierarchies</span>' | ||
| ) | ||
| _update_global_summary() |
There was a problem hiding this comment.
_load_perspective() sets each table checkbox to bool(any(vals)) after loading children. Because the table checkbox has an observer that propagates its value to all children, this turns partial selections into “all selected”. Suppress observers during this sync (e.g., keep _loading[0]=True while updating table checkboxes/icons/summaries, or temporarily unobserve/re-observe).
| members = _data.get("perspective_members", {}).get(persp_name, {}) | |
| for table_name in _data.get("tables", {}): | |
| tbl_members = members.get(table_name, {}) | |
| c_list = _child_cbs_map.get(table_name, []) | |
| for cb, obj_type, obj_name in c_list: | |
| cb.value = obj_name in tbl_members.get(obj_type, set()) | |
| _loading[0] = False | |
| # Manually sync all UI state (observers were suppressed) | |
| for table_name in _data.get("tables", {}): | |
| c_list = _child_cbs_map.get(table_name, []) | |
| vals = [cb.value for cb, _, _ in c_list] | |
| # Table checkbox | |
| if table_name in _table_cbs: | |
| _table_cbs[table_name].value = bool(any(vals)) | |
| # Status icon | |
| if table_name in _status_icons: | |
| if all(vals) and vals: | |
| _status_icons[table_name].value = ICON_ALL | |
| elif any(vals): | |
| _status_icons[table_name].value = ICON_SOME | |
| else: | |
| _status_icons[table_name].value = ICON_NONE | |
| # Table summary | |
| if table_name in _table_summary_labels: | |
| tbl_data = _data["tables"][table_name] | |
| totals = {k: len(v) for k, v in tbl_data.items()} | |
| counts = {"columns": 0, "measures": 0, "hierarchies": 0} | |
| for cb, obj_type, _ in c_list: | |
| if cb.value: | |
| counts[obj_type] += 1 | |
| _table_summary_labels[table_name].value = ( | |
| f'<span style="font-size:12px; color:{table_summary_color}; margin-left:8px;">' | |
| f'{counts["columns"]}/{totals["columns"]} columns, ' | |
| f'{counts["measures"]}/{totals["measures"]} measures, ' | |
| f'{counts["hierarchies"]}/{totals["hierarchies"]} hierarchies</span>' | |
| ) | |
| _update_global_summary() | |
| try: | |
| members = _data.get("perspective_members", {}).get(persp_name, {}) | |
| for table_name in _data.get("tables", {}): | |
| tbl_members = members.get(table_name, {}) | |
| c_list = _child_cbs_map.get(table_name, []) | |
| for cb, obj_type, obj_name in c_list: | |
| cb.value = obj_name in tbl_members.get(obj_type, set()) | |
| # Manually sync all UI state while observers are suppressed | |
| for table_name in _data.get("tables", {}): | |
| c_list = _child_cbs_map.get(table_name, []) | |
| vals = [cb.value for cb, _, _ in c_list] | |
| # Table checkbox | |
| if table_name in _table_cbs: | |
| _table_cbs[table_name].value = bool(any(vals)) | |
| # Status icon | |
| if table_name in _status_icons: | |
| if all(vals) and vals: | |
| _status_icons[table_name].value = ICON_ALL | |
| elif any(vals): | |
| _status_icons[table_name].value = ICON_SOME | |
| else: | |
| _status_icons[table_name].value = ICON_NONE | |
| # Table summary | |
| if table_name in _table_summary_labels: | |
| tbl_data = _data["tables"][table_name] | |
| totals = {k: len(v) for k, v in tbl_data.items()} | |
| counts = {"columns": 0, "measures": 0, "hierarchies": 0} | |
| for cb, obj_type, _ in c_list: | |
| if cb.value: | |
| counts[obj_type] += 1 | |
| _table_summary_labels[table_name].value = ( | |
| f'<span style="font-size:12px; color:{table_summary_color}; margin-left:8px;">' | |
| f'{counts["columns"]}/{totals["columns"]} columns, ' | |
| f'{counts["measures"]}/{totals["measures"]} measures, ' | |
| f'{counts["hierarchies"]}/{totals["hierarchies"]} hierarchies</span>' | |
| ) | |
| _update_global_summary() | |
| finally: | |
| _loading[0] = False |
PBI Fixer — Interactive UI
The main PBI Fixer application: an interactive ipywidgets-based development environment for scanning and fixing Power BI reports and semantic models directly in Microsoft Fabric Notebooks. This is the capstone PR that ties together all previous phases.
Tabs
_model_explorer.py_report_explorer.py_perspective_editor.py_vertipaq_analyzer.py_pbi_fixer.py)_pbi_fixer.py)Entry Point
Files
src/sempy_labs/_pbi_fixer.py(new file — main orchestrator)src/sempy_labs/_model_explorer.py(new file)src/sempy_labs/_report_explorer.py(new file)src/sempy_labs/_perspective_editor.py(new file)src/sempy_labs/_vertipaq_analyzer.py(new file)src/sempy_labs/__init__.py(updated — addspbi_fixerexport)Dependencies
This PR depends on:
_vertipaq.pybut works with the base version tooArchitecture Notes
ipywidgets— no custom JS extensions needed.SelectMultiplewidget serves as the tree view: single-click expands, Ctrl+click for multi-select.redirect_stdoutto prevent notebook scroll.PBI Fixer Contribution — Overview
This PR is part of the PBI Fixer contribution to semantic-link-labs — an interactive ipywidgets-based UI for scanning and fixing Power BI reports and semantic models directly in Microsoft Fabric Notebooks.
The PBI Fixer provides a tabbed ipywidgets interface (Semantic Model Explorer, Report Explorer, Perspective Editor, Vertipaq Analyzer) that lets users interactively scan, inspect, and fix Power BI artifacts without leaving the notebook. All underlying fixer functions also work as standalone API calls, so users can integrate them into scripts and pipelines without the UI.
Contribution Structure
The full contribution (~17K lines across 68 files) is split into 22 focused PRs across 6 phases to keep each PR reviewable and self-contained. Only new files are added in Phases 1–4 and 6 — no existing SLL code is modified.
/)..Find()fixes and expression capture (tom/_model.py), Vertipaq analyzer enhancements with memory/column-level analysis (_vertipaq.py, ~1000 lines changed), and various small fixes across_items.py,_item_recovery.py,_helper_functions.py,_export_report.py,_sql.py, andadmin/_tenant.py. These carry higher merge conflict risk and may need closer review or discussion.Dependencies & Review Order
sempy_labs.report.fix_piecharts(...)orsempy_labs.semantic_model.add_calculated_calendar(...).