Skip to content

fix(replay): improve CSS fidelity for multi-window, elevation, text, and border-radius#476

Open
jgibson02 wants to merge 8 commits intofix/session-replay-fidelity_issuesfrom
fix/session-replay-css-fidelity
Open

fix(replay): improve CSS fidelity for multi-window, elevation, text, and border-radius#476
jgibson02 wants to merge 8 commits intofix/session-replay-fidelity_issuesfrom
fix/session-replay-css-fidelity

Conversation

@jgibson02
Copy link
Copy Markdown

@jgibson02 jgibson02 commented Mar 18, 2026

fix(replay): improve CSS fidelity for multi-window, elevation, text, and border-radius

What & Why

This is a group of CSS fidelity fixes for this particular test app screen which highlighted the gap with layered windows.

Visual Progression

Ground Truth (Android Emulator)

Ground Truth

Before: Dialog only, no activity content

Only the topmost window (dialog) was captured. The activity behind it was missing.

Before

+ Multi-window snapshots

Activity and dialog now render together. No shadows, no scrim, sharp corners.

Multi-window

+ Elevation box-shadow

CardViews, AppBar, FAB, dialog, and buttons now have depth shadows.

Elevation

+ Dialog scrim

Semi-transparent overlay dims the activity behind the dialog.

Scrim

Final: + Text fixes, outline border-radius, container overflow

Labels no longer truncated, buttons vertically centered with sans-serif font,
CardViews and dialog have correct corner radii from outline detection, containers
clip overflow.

Final


Changes by Area

Multi-window snapshots (ViewDrawInterceptor, SessionReplayFrame, SessionReplayProcessor)

When a dialog was open, only the topmost window was captured. Each
ViewTreeObserver.OnDrawListener had its own captured decorViews array from
registration time, so no listener ever saw all windows simultaneously.

Now Curtains.getRootViews() is called at capture time inside the draw
listener to get ALL visible windows. They're combined as sibling children under
<body>. When a secondary window has FLAG_DIM_BEHIND set (dialogs), a scrim
overlay div (rgba(0,0,0,0.6)) is injected between the activity and dialog.

Elevation → box-shadow (ViewDetails)

View.getElevation() is mapped to CSS box-shadow using a Material Design
approximation: 0px {e*0.5}px {e}px rgba(0, 0, 0, 0.24).

Outline-based border-radius (ViewDetails)

ViewOutlineProvider is queried at capture time to extract corner radii for
views that clip to their outline (clipToOutline==true) but don't use
GradientDrawable backgrounds (dialogs, CardViews, FABs).

Container overflow (ViewDetails)

ViewGroup.getClipChildren() drives overflow: hidden on containers, fixing
text overflow in clipped views like CardViews.

Text CSS fixes (SessionReplayTextViewThingy)

  • Fixed double semicolons in getFontFamily() that broke CSS parsing
  • Use white-space: nowrap for single-line TextViews (maxLines==1),
    pre-wrap for multi-line text to preserve wrapping
  • Added flexbox vertical centering for views with CENTER_VERTICAL gravity
  • Added font-family: sans-serif for null-typeface views (e.g. buttons)
  • Added line-height from TextView.getLineHeight()

Incremental diff safety (SessionReplayProcessor)

Simplified the snapshot decision: incremental diff only runs for single-window
frames with matching root view IDs and viewport dimensions. All other cases
produce a full snapshot.

Callouts

  • Scrim is only injected when FLAG_DIM_BEHIND is set — popups, toasts, and
    menus don't get a false dimming overlay
  • Outline radius is only used when clipToOutline==true — views that use
    outline for shadow shape only won't get false border-radius
  • Scrim dimAmount is hardcoded to 0.6 (Android default) — reading
    WindowManager.LayoutParams.dimAmount at capture time is a follow-up
  • All String.format calls use Locale.US to prevent locale-dependent
    decimal separators in CSS values

Test plan

  • Build agent locally, install on test app
  • Dashboard screen: verify CardView rounded corners, elevation shadows,
    overflow clipping, text not truncated
  • Open dialog: verify scrim overlay, dialog border-radius, multi-window
    snapshot (activity + dialog visible), button vertical centering and
    sans-serif font
  • Compare rrweb replay against emulator screenshot

@jgibson02 jgibson02 self-assigned this Mar 18, 2026
jgibson02 added a commit that referenced this pull request Mar 18, 2026
@jgibson02 jgibson02 marked this pull request as ready for review March 18, 2026 16:22
@jgibson02 jgibson02 force-pushed the fix/session-replay-css-fidelity branch from d952a41 to ba6d3b3 Compare March 19, 2026 01:00
@ndesai-newrelic ndesai-newrelic force-pushed the fix/session-replay-fidelity_issues branch from 098e2a8 to 690b4d0 Compare March 19, 2026 17:21
…and border-radius

- Capture all visible windows via Curtains.getRootViews() so dialogs
  render with the activity behind them instead of on a blank background
- Inject a scrim overlay (rgba(0,0,0,0.6)) between the activity and
  dialog windows to replicate Android's FLAG_DIM_BEHIND behavior
- Map View.getElevation() to CSS box-shadow for Material Design depth
- Extract corner radius from ViewOutlineProvider for dialogs, CardViews,
  and FABs that don't use GradientDrawable backgrounds
- Emit overflow:hidden on ViewGroup containers based on clipChildren
- Fix double semicolons in getFontFamily() that broke CSS parsing
- Replace blanket overflow:hidden on text views with white-space:nowrap
  to prevent false truncation from font metric differences
- Add flexbox vertical centering for views with CENTER_VERTICAL gravity
- Include font-family:sans-serif for null-typeface views (e.g. buttons)
- Add line-height CSS property from TextView.getLineHeight()
- Use Locale.US in String.format for CSS numeric values
- Simplify incremental-vs-full snapshot decision in processFrames
…radius

- Only inject dialog scrim when FLAG_DIM_BEHIND is set on the window,
  preventing false dimming behind popups, toasts, and menus
- Use white-space:nowrap only for single-line TextViews (maxLines==1),
  fall back to pre-wrap for multi-line text to preserve wrapping
- Only extract outline radius when view.getClipToOutline() is true,
  preventing border-radius on views that use outline for shadow shape
  only without visual clipping
@jgibson02 jgibson02 force-pushed the fix/session-replay-css-fidelity branch from ba6d3b3 to 9b0a09f Compare March 30, 2026 19:42
@ndesai-newrelic ndesai-newrelic force-pushed the fix/session-replay-fidelity_issues branch from 1cf2428 to e442da0 Compare March 31, 2026 17:16
@ywang-nr
Copy link
Copy Markdown
Contributor

Waiting for performance testing

@ndesai-newrelic ndesai-newrelic force-pushed the fix/session-replay-fidelity_issues branch 2 times, most recently from a38dddf to ad7afe4 Compare March 31, 2026 21:14
jgibson02 and others added 2 commits April 9, 2026 14:05
…fidelity

Resolve conflicts:
- SessionReplayProcessor.java: keep Attributes import (HEAD) + NewRelicIdGenerator/SessionReplayViewThingyInterface imports (base)
- ComposeBlockedViewThingy.java: use updated viewMapper.SessionReplayViewThingyInterface path (base)
- ViewTouchHandler.java: keep superset of imports from base (includes HEAD's ComposeSessionReplayConstants)
- ViewDetails.java: keep Outline/Paint/Field/Locale imports needed for CSS fidelity code (HEAD)
- performance.md: take consolidated Round 2 data with updated terminology (base)
@ndesai-newrelic
Copy link
Copy Markdown
Contributor

@diegomtz5 we need test this on lambada test with multi devices.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants