Skip to content

Add Ren'Py tokenizer and text formatting display#240

Merged
mikkisguy merged 3 commits into
mainfrom
137-design-renpy-text-formatting-display-in-prose-editor
Jun 18, 2026
Merged

Add Ren'Py tokenizer and text formatting display#240
mikkisguy merged 3 commits into
mainfrom
137-design-renpy-text-formatting-display-in-prose-editor

Conversation

@mikkisguy

@mikkisguy mikkisguy commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • New Features

    • Click-to-edit: Click directly on formatted dialogue text to position your cursor
    • Live formatting preview: Dialogue text now displays with applied styles (bold, italic, colors, underline, size) in the editor
  • Tests

    • Added comprehensive test coverage for dialogue editing and text formatting features

…'Py tag tokenizer

- Enhanced DialogueLine tests to cover textarea visibility and interaction.
- Introduced RenderedLine tests for rendering formatted text and handling tags.
- Implemented a Ren'Py tag tokenizer to parse dialogue text with inline tags.
- Added unit tests for the tokenizer to ensure correct tokenization and round-trip functionality.
@mikkisguy mikkisguy self-assigned this Jun 18, 2026
@mikkisguy mikkisguy linked an issue Jun 18, 2026 that may be closed by this pull request
5 tasks
@vercel

vercel Bot commented Jun 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
branchforge-docs Ready Ready Preview, Comment Jun 18, 2026 8:08pm

@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

React Doctor found no issues. 🎉

Reviewed by React Doctor for commit f12b9c3.

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@mikkisguy, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 25 minutes and 6 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3c23d796-c1c9-4229-b73a-e9ff010d9ff1

📥 Commits

Reviewing files that changed from the base of the PR and between 9af77a2 and f12b9c3.

📒 Files selected for processing (2)
  • apps/frontend/src/components/write-mode/__tests__/DialogueLine.test.tsx
  • apps/frontend/src/test/setup.ts
📝 Walkthrough

Walkthrough

Adds a new renpy-tags.ts lossless tokenizer for Ren'Py dialogue markup, a new RenderedLine component that renders token streams as styled spans with raw-offset data attributes, and rewrites DialogueLine to show a formatted overlay when blurred and map overlay clicks back to textarea caret positions when focused.

Changes

Ren'Py tokenizer, RenderedLine component, and DialogueLine overlay

Layer / File(s) Summary
Ren'Py tokenizer types, scanner, and helpers
apps/frontend/src/lib/renpy-tags.ts, apps/frontend/src/lib/renpy-tags.unit.test.ts
Exports RenpyToken discriminated union, RenpyTag/RenpySelfTag name unions, tokenize() single-pass scanner with escape/interpolation/tag/malformed handling, stringify() round-trip serializer, and internal parsing helpers. Unit tests cover all tag forms, escape sequences, interpolation, malformed/unknown tags, lossless round-trips, and compile-time type checks.
RenderedLine component
apps/frontend/src/components/write-mode/RenderedLine.tsx, apps/frontend/src/components/write-mode/__tests__/RenderedLine.test.tsx
New RenderedLine component renders RenpyToken[] as styled <span> nodes with data-raw-start/data-raw-len attributes for raw-offset mapping. Style helpers (computeStyle, sizeToFontSize, alphaToOpacity, popMatchingTag) derive CSS from the active tag stack. Tests validate rendering, tag hiding, style application, interpolation, malformed tokens, size/alpha, newlines, and edge cases.
DialogueLine overlay/focus split and click-to-edit
apps/frontend/src/components/write-mode/DialogueLine.tsx, apps/frontend/src/components/write-mode/__tests__/DialogueLine.test.tsx
Adds getRawOffsetFromPoint using caretPositionFromPoint/caretRangeFromPoint and span metadata to map overlay clicks to raw textarea offsets. Introduces isFocused state, handleRenderedLineClick callback, and memoized renderedTokens. Replaces always-visible textarea with a split UI: textarea hidden when not focused, formatted RenderedLine overlay shown instead. Tests verify overlay/textarea visibility transitions on click and blur.

Sequence Diagram(s)

sequenceDiagram
  actor User
  participant OverlayButton as Overlay Button (RenderedLine)
  participant getRawOffsetFromPoint
  participant DialogueLine
  participant Textarea

  User->>OverlayButton: click at (x, y)
  OverlayButton->>getRawOffsetFromPoint: clientX, clientY
  getRawOffsetFromPoint->>getRawOffsetFromPoint: caretPositionFromPoint / caretRangeFromPoint
  getRawOffsetFromPoint->>getRawOffsetFromPoint: read data-raw-start, data-raw-len from span
  getRawOffsetFromPoint-->>DialogueLine: rawOffset
  DialogueLine->>Textarea: focus()
  DialogueLine->>Textarea: setSelectionRange(rawOffset, rawOffset)
  DialogueLine->>DialogueLine: setIsFocused(true) → hide overlay, show textarea

  User->>Textarea: blur
  Textarea->>DialogueLine: onBlur
  DialogueLine->>DialogueLine: setIsFocused(false) → show overlay, hide textarea
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐇 Hop, hop, through the tags I go,
{b}bold{/b} and {i}italic{/i} all in a row,
A click on the overlay, a caret appears,
Raw offsets and spans banish all fears.
The textarea hides, the tokens shine bright —
This rabbit tokenized everything right! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The pull request title accurately describes the main changes: a Ren'Py tokenizer and text formatting display component system added across multiple new files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 137-design-renpy-text-formatting-display-in-prose-editor

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot changed the title @coderabbitai Add Ren'Py tokenizer and text formatting display Jun 18, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/frontend/src/components/write-mode/__tests__/DialogueLine.test.tsx`:
- Around line 95-138: The test "focuses the textarea when the rendered line is
clicked" verifies that clicking the overlay enters edit mode and shows the
textarea, but it does not validate that the caret position (selectionStart and
selectionEnd) is correctly mapped to the click position on the overlay. Add
assertions after the user.click call on the data-rendered-line-wrapper element
to verify that the textarea's selectionStart and selectionEnd properties are set
to the expected offset corresponding to the click position, ensuring that
getRawOffsetFromPoint correctly translates overlay coordinates to raw text
offsets.

In `@apps/frontend/src/components/write-mode/__tests__/RenderedLine.test.tsx`:
- Around line 10-11: Update the import statements in the file to comply with ESM
guidelines by adding `.js` extensions and using the `@/` alias. For the tokenize
import from "`@/lib/renpy-tags`", add the `.js` extension to make it
"`@/lib/renpy-tags.js`". For the RenderedLine import that currently uses the
relative path "../RenderedLine", convert it to use the `@/` alias pointing to
the appropriate path within the components directory and add the `.js` extension
(e.g., "`@/components/write-mode/RenderedLine.js`").

In `@apps/frontend/src/components/write-mode/DialogueLine.tsx`:
- Around line 18-24: Convert all relative imports in the DialogueLine.tsx import
statement block to use the `@/` alias convention and add `.js` extensions to all
import paths. Specifically, change the relative imports like
`"./TechnicalBadge"`, `"./TechnicalPopover"`, and `"./RenderedLine"` to their
equivalent `@/` paths (such as `@/components/write-mode/TechnicalBadge.js`), and
add `.js` extensions to all other import paths including those already using
`@/` aliases (for example, `@/lib/prose-types.js`, `@/lib/utils.js`, and
`@/lib/renpy-tags.js`). Ensure all import statements follow the project's
frontend import conventions.

In `@apps/frontend/src/lib/renpy-tags.unit.test.ts`:
- Line 2: The import statement for tokenize, stringify, and RenpyToken from the
local module is missing the required ESM file extension. Update the import path
in the import statement from "`@/lib/renpy-tags`" to include the ".js" extension
at the end, changing it to "`@/lib/renpy-tags.js`" to comply with the ESM module
resolution requirements.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7531d3d8-f51e-4849-849d-d9033a941424

📥 Commits

Reviewing files that changed from the base of the PR and between c648b0d and 9af77a2.

📒 Files selected for processing (6)
  • apps/frontend/src/components/write-mode/DialogueLine.tsx
  • apps/frontend/src/components/write-mode/RenderedLine.tsx
  • apps/frontend/src/components/write-mode/__tests__/DialogueLine.test.tsx
  • apps/frontend/src/components/write-mode/__tests__/RenderedLine.test.tsx
  • apps/frontend/src/lib/renpy-tags.ts
  • apps/frontend/src/lib/renpy-tags.unit.test.ts

Comment thread apps/frontend/src/components/write-mode/DialogueLine.tsx
Comment thread apps/frontend/src/lib/renpy-tags.unit.test.ts
@mikkisguy mikkisguy merged commit 9a2ee1d into main Jun 18, 2026
12 checks passed
@mikkisguy mikkisguy deleted the 137-design-renpy-text-formatting-display-in-prose-editor branch June 18, 2026 20:12
@mikkisguy mikkisguy restored the 137-design-renpy-text-formatting-display-in-prose-editor branch June 20, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design: Ren'Py text formatting display in Prose Editor

1 participant