Skip to content

feat(export): add Notion, Bilibili, Mastodon, Bluesky, Markdown round-trip targets - #19

Merged
nettee merged 4 commits into
nexu-io:mainfrom
wuwangzhang1216:feat/export-targets
May 19, 2026
Merged

feat(export): add Notion, Bilibili, Mastodon, Bluesky, Markdown round-trip targets#19
nettee merged 4 commits into
nexu-io:mainfrom
wuwangzhang1216:feat/export-targets

Conversation

@wuwangzhang1216

@wuwangzhang1216 wuwangzhang1216 commented May 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds five new export targets under src/lib/export/, filling out the matrix beyond WeChat / Zhihu / image / clipboard / download / deck:

  • notion.ts — juice-inlined fragment for Notion paste. Drops <section>/<article> wrappers (Notion unwraps anyway) and strips class / data-* noise, but preserves language-* on <code> so Notion picks the right code-block mode.
  • bilibili.ts — 专栏-compatible HTML subset. DOM walker through a tag whitelist (p, h1-6, lists, pre, code, a, img, figure, span, etc.). Images not on *.hdslb.com / *.bilibili.com get swapped for a placeholder SVG with data-original-src stashed for recovery.
  • mastodon.ts / bluesky.ts — screenshot + short-caption share bundles. Reuses iframeToBlob for the image. Captions are code-point-truncated to 500 / 300 chars (no surrogate-pair / emoji slicing). Image and text are pushed through separate clipboard calls because browsers reject mixed-mime writes.
  • markdown-roundtrip.ts — focused HTML → GFM walker for Hugo / 11ty / Obsidian users. Handles headings, nested lists, fenced code with language, tables, links, images, blockquotes, hr, br. Skipped pulling in turndown.

Notes

  • Wire-up into export-menu.tsx is intentionally left for a follow-up so this PR stays mechanical/reviewable.
  • tsc --noEmit clean.

Test plan

  • Paste output from toNotionHtml into a Notion doc — verify headings, code blocks (with language), and inline color/weight survive
  • Paste output from toBilibiliHtml into a 专栏 draft — confirm unsupported tags are gone and external images render as the placeholder
  • buildMastodonShare / buildBlueskyShare against a sample preview iframe — verify caption truncation respects code points (try one with emoji past the limit)
  • htmlToMarkdown on a deck-style document — verify nested lists, code fences with language hints, and tables render correctly in a Hugo preview

@wuwangzhang1216

Copy link
Copy Markdown
Contributor Author

Pushed 8a40fe2 addressing the review feedback.

markdown-roundtrip.ts

  • Inline code with backtickswrapInlineCode now picks a fence longer than the longest backtick run in the body and pads with a space when the content starts/ends with ` (CommonMark rule). Backslash-escape attempt is gone.
  • **Fenced code with embedded ** — `renderPre` uses the same `backtickFence(body, 3)` helper so a fenced block containing no longer terminates early.
  • Link / image escapingescapeHref wraps URLs with whitespace or parens in <…> (percent-encoding any literal </>); escapeTitle handles " and \ inside titles; escapeBrackets escapes [/] in link text and img alt without double-escaping the \ from prior escapeMd passes.
  • Line-starting block markers — new escapeBlockStarts pass on paragraph + fallback inline-at-block content escapes leading #, >, -, +, *, digits. only when followed by whitespace/EOL, so a paragraph starting with 1. First no longer renders as an ordered list.
  • Block content inside <li>renderList now partitions list-item children into inline / nested-list / other-block buckets. Block children (paragraphs, code blocks, blockquotes, tables) are rendered through renderBlock with a continuation indent matching the marker width, so list items with embedded paragraphs or fenced code nest cleanly.
  • escapeMd now also escapes ~ so stray ~~text~~ in plain prose doesn't become strikethrough.

bilibili.ts

  • <code class> is filtered to keep only language-* tokens; arbitrary highlighter classes (hljs-keyword, etc.) are dropped.
  • Removed the dead "Wrap loose inline children — caller decides" comment in the unwrap branch since no caller actually does that.

mastodon.ts

  • Comment on truncateForPost now explicitly notes the function counts code points, not graphemes. ZWJ-joined emoji like 👨‍👩‍👧 count as 5 here instead of 1 — conservative, never overshoots the platform limit, but compound-emoji-heavy captions get trimmed more aggressively than strictly necessary.

tsc --noEmit is still clean. Tests deferred along with the menu wire-up.

@wuwangzhang1216

Copy link
Copy Markdown
Contributor Author

Pushed b48e5b3 — adds 50 vitest tests for the new exporters in addition to the review fixes from the prior comment.

Test approach: "simulate real"

  • happy-dom as the test environment so the exporters run against a real DOMParser / Element / TreeWalker (the same APIs the prod code uses at runtime).
  • For markdown-roundtrip, every emitted Markdown sample is piped back through marked (already a runtime dependency for the editor's preview) and the resulting HTML is asserted to round-trip the structural intent. Closest "real" check possible without standing up a Hugo / 11ty / Obsidian preview.
  • For mastodon.truncateForPost, tested with real surrogate pairs and a ZWJ-joined family emoji.

Coverage

File Tests Highlights
markdown-roundtrip.test.ts 25 Inline code containing backticks; fenced code with embedded ```; links with spaces/parens in href using the angle-bracket form; title quote escape; bracket escape in link text; paragraphs starting with `1.` / `#` / `>` / `-` staying as paragraphs; nested lists; list items with paragraph + code block; pipe escape in tables; full-document smoke.
bilibili.test.ts 9 <script>/<iframe>/<style> stripped; <div>/<section> unwrapped; class/data-*/onclick dropped; javascript: href dropped; language-* class retained on <code>; CDN vs. non-CDN image rewrite.
notion.test.ts 8 <section>/<article>/<header>/<footer>/<main> unwrap; class/data-* strip; inline style preserved; <code> language-* retention; language-plaintext fallback; <pre> without <code> gets wrapped.
mastodon.test.ts 8 Whitespace collapse, surrogate-pair safety, ZWJ-emoji conservative count, boundary cases.

pnpm test wired in package.json. tsc --noEmit clean.

Extra fixes uncovered while writing tests

  • renderList: when an <li> has no inline children (e.g. <li><p>x</p><pre>…</pre></li>), the first <p>'s content is hoisted onto the marker line. Without this, marked saw 1. \n\n text and treated the next marker as a fresh list.
  • notion.ts <code class> filter now keeps only language-* tokens (matches bilibili.ts); previously hljs / token-keyword / etc. could leak through.

Test plan items 1–2 (Notion / Bilibili paste) and item 4 (Hugo preview) still need manual verification with a real account, but the suite covers the underlying transformation logic that drives all three. Item 3 (buildMastodonShare / buildBlueskyShare) depends on iframeToBlob which needs a real iframe; the caption-truncation half is covered by the mastodon.test.ts suite.

@lefarcen
lefarcen requested a review from PerishCode May 15, 2026 13:12
@lefarcen lefarcen added size/XXL PR size: 1500+ changed lines risk/high High-risk PR: dependencies, infra, security-sensitive, or broad runtime impact type/feature Feature or new user-facing capability labels May 15, 2026

@PerishCode PerishCode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wuwangzhang1216 nice clean set of new emitters — the test matrices (DOMParser round-trip + marked validation for the Markdown emitter) are exactly the right level of coverage and caught the Unicode/codepoint edge cases I would have asked about. A few non-blocking correctness notes inline on three edge cases that the current fixtures don't cover yet: a CDN allowlist regex that admits lookalike domains, and two Markdown round-trip paths (image/code inside <a>, and <br> inside table cells) that the emitter silently mangles. Happy to land as-is if you'd rather follow up separately.

🔁 Powered by Looper · runner=reviewer · agent=claude-code · An autonomous AI dev team for your GitHub repos.

Comment thread src/lib/export/bilibili.ts Outdated
Comment thread src/lib/export/markdown-roundtrip.ts Outdated
Comment thread src/lib/export/markdown-roundtrip.ts Outdated
@lefarcen
lefarcen requested a review from qiongyu1999 May 15, 2026 13:26
@wuwangzhang1216

Copy link
Copy Markdown
Contributor Author

Pushed d786299 — all three non-blocking findings from @PerishCode's review are addressed, with regression tests for each.

1. isBilibiliCdn lookalike-domain fix

bilibili.ts

  • New regex: ^https?:\/\/(?:[^/]+\.)?(?:hdslb\.com|bilibili\.com)(?:[/?#]|$)/i
  • Either an exact host match or a subdomain dot is required before the literal — evilhdslb.com, notbilibili.com, etc. now fall through to the placeholder swap.
  • Tests in bilibili.test.ts:
    • rejects lookalike hostnames — asserts data-bili-placeholder="true" is set for https://evilhdslb.com/foo.png and https://notbilibili.com/foo.png.
    • accepts subdomains of the bilibili CDN — verifies i0.hdslb.com, album.bilibili.com, and bare apex hdslb.com / bilibili.com all pass through untouched.

2. <img> / <code> nested inside <a> no longer mangled

markdown-roundtrip.ts

  • Took option A from the suggestion: moved [ / ] escape into escapeMd (text-node level) and dropped the post-composition escapeBrackets(inner()) wrapper. escapeBrackets helper deleted.
  • Nested rendered structure now flows into […](href) untouched, so <a><img alt=a src=y></a> becomes [![a](y)](href) and round-trips back into <a><img> correctly.
  • Tests in markdown-roundtrip.test.ts:
    • preserves <img> nested inside <a> (badge-style link) — asserts the round-tripped DOM has <a> containing an <img> with the original src/alt.
    • preserves <code> nested inside <a> — asserts the <code> survives and contains no stray backslashes inside its textContent.
  • Existing escapes [ and ] inside link text test still passes — text-node bracket escape covers that case too.

3. Table cells with <br> no longer break the table

markdown-roundtrip.ts renderTable

  • Cells now collapse \r?\n+ to literal <br> (inline HTML is allowed inside GFM cells) before pipe-escaping, so a literal \n from a <br> / <p> no longer terminates the table row early.
  • Test in markdown-roundtrip.test.ts:
    • keeps a cell containing <br> on a single row<table><tr><th>H</th></tr><tr><td>line1<br>line2</td></tr><tr><td>next</td></tr></table> round-trips with 3 <tr>s (header + 2 body), and the first body cell contains both text fragments plus a <br>.

pnpm test55/55 green (was 50). tsc --noEmit clean.

@lefarcen

Copy link
Copy Markdown

Thanks for the detailed fix notes, @wuwangzhang1216 — that makes the follow-up easy to track.

@PerishCode could you re-check the current head when you have a moment? The author says the three non-blocking findings from your review are addressed with regression coverage. @qiongyu1999, your product approval is now stale after the push as well, so a quick current-head re-approval would unblock the product gate once review/mergeability are otherwise ready.

…trip targets

- notion.ts: juice-inline body + drop section wrappers and class/data-* noise;
  keep `language-*` on <code> so Notion picks the right code mode.
- bilibili.ts: walk DOM through a 专栏 tag whitelist; replace non-bili-CDN
  images with a "re-upload to bilibili" placeholder, stash the original src
  on data-original-src so the user can recover it.
- mastodon.ts / bluesky.ts: build {blob, text, filename} share bundles —
  screenshot via the existing iframeToBlob, caption trimmed by code-point
  count to 500 / 300 chars (so emoji and surrogate pairs don't get sliced).
  Image and text go on the clipboard via separate calls — browsers refuse
  mixed-mime writes.
- markdown-roundtrip.ts: focused HTML → GFM walker covering the subset the
  editor renders (headings, lists with nesting, fenced code with language,
  tables, links, images, blockquotes). Skipped turndown to avoid pulling a
  large dep for a narrow input.

Wire-up into the export menu is left for a follow-up so this PR stays
mechanical.
…comment

Addresses review feedback on the new export targets:

- markdown-roundtrip: use longest-backtick-run fences for inline + fenced
  code so embedded backticks don't break the block; URL-escape link/image
  hrefs with whitespace or parens via the angle-bracket form, and escape
  quotes/backslashes in titles; backslash-escape brackets inside link
  text and img alt; new escapeBlockStarts pass keeps a paragraph that
  starts with "# ", "> ", "- ", "1. ", etc. from being parsed as a heading
  or list; renderList now partitions <li> children into inline / nested-
  list / other-block buckets so paragraphs and code blocks inside list
  items render with a proper continuation indent.
- bilibili: filter <code class> to language-* only so arbitrary highlight
  classes don't leak through; drop dead "caller decides" comment.
- mastodon: clarify that truncateForPost counts code points, not graphemes
  — ZWJ emoji sequences are conservatively counted as multiple units.
…mitters

Adds vitest + happy-dom and 50 tests covering the new export targets.
Each test runs in a real DOM environment; the markdown emitter tests
additionally pipe the output through `marked` (already a runtime dep)
to verify the produced Markdown round-trips into the same structural
shape — closest "simulate real" check we can do without spinning up
Hugo / 11ty / Obsidian.

- markdown-roundtrip.test.ts (25): headings, inline marks (with literal
  *, _, ~ kept literal), inline code with backticks (longest-fence
  rule), fenced code with embedded triple backticks, links/images
  (angle-bracket URL form for spaces+parens, title quote escape,
  bracket escape in link text), line-start block-marker escaping
  (paragraph starting with `1. `, `# `, `- `, `> ` stays a paragraph),
  nested lists, list item with paragraph + code block, blockquote,
  table with pipe escape, hr, full-document smoke.
- bilibili.test.ts (9): whitelist enforcement, attribute filtering,
  language-* class retention on <code>, CDN vs. non-CDN image rewrite.
- notion.test.ts (8): section/article/header/footer/main unwrapping,
  data-* + class stripping, style attribute preserved, code language-*
  class retention, language-plaintext fallback, <pre> without <code>
  gets wrapped.
- mastodon.test.ts (8): whitespace collapse, surrogate-pair safety,
  ZWJ-emoji conservative behavior, boundary cases.

While writing these:
- Hoisted the first <p>'s inline content onto the list-marker line when
  an <li> has no inline children; otherwise marked saw "1. \n\n   text"
  and started a new list at the next marker.
- Tightened notion.ts <code class> filter to keep only language-*
  tokens (matches bilibili.ts; arbitrary highlighter classes no longer
  leak through).

`pnpm test` (now wired in package.json) runs the full suite; `tsc
--noEmit` clean.
Addresses the three non-blocking findings from PerishCode's review:

- bilibili `isBilibiliCdn`: tighten the regex so `evilhdslb.com` /
  `notbilibili.com` no longer pass as valid CDNs. Require either an
  exact host match or a subdomain dot before the literal domain.
  Without this, the placeholder swap was skipped for lookalike URLs
  and the user would silently lose the image after bilibili's
  publish-time scrub.

- markdown emitter, nested elements inside <a>: move bracket escape
  into `escapeMd` at the text-node level instead of wrapping the
  fully-composed `inner()` string. The old approach mangled
  <a><img src=y alt=a></a> to `[!\[a\](y)](href)` so the inner
  image no longer round-tripped; same for `<a><code>x</code></a>`
  where literal backslashes appeared inside the code span.

- renderTable: GFM tables require one row per source line, so a
  literal \n inside a cell (from <br>, <p>, etc.) terminates the
  row early and collapses every downstream row. Collapse intra-cell
  \r?\n+ to literal `<br>` (inline HTML is permitted inside GFM
  cells) before pipe-escaping.

Tests added: bilibili lookalike + subdomain matrix; markdown emitter
`<a><img>` (badge link), `<a><code>` (no stray backslashes), and
`<td>line1<br>line2</td>` (table stays intact). 55/55 green.
@PerishCode
PerishCode force-pushed the feat/export-targets branch from d786299 to 3a0f402 Compare May 18, 2026 12:44
@PerishCode

Copy link
Copy Markdown
Contributor

Thanks for the PR and the follow-up fixes. I rebased this branch onto the latest main after the public workspace / CI harness landed.

What changed:

  • Moved the new exporter modules from src/lib/export/... to next/src/lib/export/....
  • Moved the Vitest coverage into next/src/lib/export/__tests__/... so it runs through the Next package test command.
  • Kept the root package as workspace metadata only; no root scripts or root vitest.config.ts were reintroduced.
  • Kept the feature scope library-only: the new targets are still not wired into ExportMenu, matching the original PR intent.

Validation:

  • pnpm install --frozen-lockfile
  • pnpm exec tsx scripts/guard.ts
  • pnpm -F @html-anything/next typecheck
  • pnpm -F @html-anything/e2e typecheck
  • pnpm -F @html-anything/next test — 59 tests passed
  • pnpm -F @html-anything/next build
  • pnpm -F @html-anything/e2e test — 3 tests passed
  • git diff --check

Risk: Low-to-medium. The rebase was limited to directory/package-shape alignment, and the exporter test suite stayed green. Product wiring for these targets remains out of scope for this PR.

@lefarcen lefarcen added size/XL PR size: 700-1499 changed lines risk/medium Medium risk change and removed size/XXL PR size: 1500+ changed lines risk/high High-risk PR: dependencies, infra, security-sensitive, or broad runtime impact labels May 18, 2026
@wuwangzhang1216

Copy link
Copy Markdown
Contributor Author

Thanks @PerishCode for taking the time to rebase this onto the new next/ workspace layout — much appreciated. I pulled the rebased branch locally and re-ran the full validation matrix to confirm nothing regressed in transit:

  • pnpm exec tsx scripts/guard.ts — passed
  • pnpm -F @html-anything/next typecheck — clean
  • pnpm -F @html-anything/next test59/59 passed
  • pnpm -F @html-anything/next build — succeeded

I've also gone through and resolved the three outdated review threads from the original review pass. Each of the findings landed in 3a0f402 with regression coverage:

  • isBilibiliCdn lookalike-hostname tightening — covered by the rejects lookalike hostnames / accepts subdomains of the bilibili CDN cases in bilibili.test.ts.
  • Bracket-escape mangling of <img> / <code> nested inside <a> — covered by the preserves <img> nested inside <a> and preserves <code> nested inside <a> cases in markdown-roundtrip.test.ts.
  • Table-cell <br> collapsing into a literal newline — covered by keeps a cell containing <br> on a single row.

With those threads closed, the PR is now APPROVED + CLEAN and the ruleset gates are satisfied. Ready for a squash-merge whenever it's convenient — thanks again to both reviewers for the careful pass.

@nettee
nettee merged commit 122b780 into nexu-io:main May 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk/medium Medium risk change size/XL PR size: 700-1499 changed lines type/feature Feature or new user-facing capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants