Merge pull request #76 from llblab/dev

0.17.1: Rich Markdown parser hotfix

Author: llblab

AGENTS.md

@@ -68,14 +68,14 @@ The core product loop is mobile continuation: start or supervise work in the ter
 - Abort-history mode is scoped to Telegram-owned active turns. Internally this is `foldQueuedPromptsIntoHistory`: local/non-Telegram agent starts after abort clear it so the next Telegram prompt appends to the local queue instead of absorbing older queued turns as history
 - `/start`, `/help`, and `/status` open the unified command-help/status-row/control menu; `/model`, `/thinking`, and `/queue` jump to sections directly; visible bot commands are `/start`, `/compact`, `/next`, `/continue`, `/abort`, `/stop`
 - Command/menu emoji are fixed UI adornments owned by the `commands` map; do not add a persisted emoji toggle or Settings menu until there is a real setting to own
-- Telegram `reply_to_message` context is prompt-only and must not affect slash-command parsing
+- Telegram `reply_to_message` context is prompt-only and must not affect slash-command parsing; when Telegram includes `rich_message` blocks on a quoted rich reply, extract plain text from those blocks before falling back to raw `text`/`caption` so prompt context does not leak raw Rich Markdown source
 - Long-lived timers, pollers, watchers, and deferred queue dispatch must be session-bound and avoid stale live π contexts after session replacement
 - Do not add Telegram commands that imitate Pi interactive session replacement, navigation, or TUI rendering through private internals, ANSI terminal clearing, raw TTY injection, or a shadow `pi` subprocess. Features such as a real Telegram `/new` require a public Pi API that runs the same session-replacement path as the terminal command.
 - In-flight `/model` switching is limited to Telegram-owned active turns; if a tool call is active, abort is delayed until the tool finishes
 
 ## 5.3 Telegram Delivery Semantics
 
-- Assistant and guest replies use Telegram-native Rich Markdown via Rich Message APIs, not Markdown→HTML conversion. Bridge-owned UI surfaces such as commands, menus, status, queue controls, and sections should keep explicit Telegram HTML/plain rendering by default because readability and maintainability are higher there. Companion sections may explicitly choose Markdown, HTML, or plain text per view
+- Assistant and guest replies use Telegram-native Rich Markdown via Rich Message APIs, not Markdown→HTML conversion. Bridge-owned UI surfaces such as commands, menus, status, queue controls, and sections should keep explicit Telegram HTML/plain rendering by default because readability and maintainability are higher there. Companion sections may explicitly choose Markdown, HTML, or plain text per view. Keep native Rich Markdown source close to model-authored Markdown, but normalize Bot-API-fragile equivalents when evidence shows a Telegram parser/client edge, such as space-after-marker blockquotes (`> quote` -> `>quote`) and dollar-prefixed ticker atoms (`$BLDR` -> `\$BLDR`) outside code fences/spans
 - Use `docs/telegram-bot-api-rich-messages.md` as the local Bot API/Rich Messages reference for native Rich Markdown work
 - Formula guidance belongs in the Telegram-turn prompt contract: use `$...$` for inline math and `$$...$$` for block math; backticks intentionally render formulas as literal code
 - Real code blocks must stay literal and escaped

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ## Unreleased
 
+## 0.17.1: Rich Markdown Parser Hotfix
+
+- `[Rich Markdown]` Normalize Bot-API-fragile source before native Rich Markdown delivery, including space-after-marker blockquotes and dollar-prefixed ticker atoms such as `$BLDR` / `$NTVE`, prefer Telegram `rich_message` blocks over raw `text`/`caption` when extracting quoted reply context for prompts, and keep Telegram copyability guidance generic by recommending inline code for short copyable literals. Impact: assistant replies are less likely to render as raw Markdown on Telegram Rich Message parser/client edges, replying to a native Rich Markdown bot message no longer injects raw Markdown source into `[reply]`, and prompts nudge copyable identifiers without ticker-specific bloat.
+
 ## 0.17.0: Native Rich Markdown Delivery
 
 - `[Rich Markdown]` Assistant and guest replies now use Telegram-native Rich Message APIs directly: final assistant Markdown goes through `sendRichMessage`, streaming drafts go through `sendRichMessageDraft`, editable fallback previews finalize through `editMessageText.rich_message`, and guest replies use `InputRichMessageContent`. Impact: model-authored Markdown reaches Telegram as native Rich Markdown instead of passing through the legacy Markdown-to-HTML assistant path.

docs/architecture.md

@@ -52,7 +52,7 @@ The repository uses a **Flat Domain DAG**:
 - `config` / `setup`: `telegram.json`, bot token setup, first-user pairing, authorization, env fallback, atomic persistence, and live config accessors.
 - `locks` / `polling`: singleton polling ownership, takeover/restart behavior, long-poll controller state, offset persistence, and poll-loop wiring.
 - `updates` / `routing`: update classification, authorization planning, callbacks, edited messages, reactions, and inbound route composition.
-- `media` / `text-groups` / `time-injection` / `turns` / `inbound`: inbound text/media/file extraction, media-group debounce, long-text coalescing, optional `[time]` context, handler execution, and prompt-turn assembly/editing.
+- `media` / `text-groups` / `time-injection` / `turns` / `inbound`: inbound text/media/file extraction, rich-message reply-context plaintext recovery, media-group debounce, long-text coalescing, optional `[time]` context, handler execution, and prompt-turn assembly/editing.
 - `queue`: queue item contracts, lane admission/order, readiness gates, mutations, dispatch runtime, prompt/control enqueueing, and session/agent/tool lifecycle sequencing.
 - `runtime`: session-local coordination primitives: counters, flags, setup guard, abort handler, typing timers, dispatch flags, and reset binding.
 - `model` / `menu-model` / `menu-thinking` / `menu-status` / `menu-queue` / `menu-settings` / `menu` / `commands`: model identity, thinking levels, scoped model handling, menu render/callback behavior, slash commands, bot commands, and interactive controls.
@@ -196,6 +196,8 @@ Assistant replies use Telegram-native Rich Markdown. Final Markdown is sent dire
 Assistant delivery guarantees:
 
 - Model-authored Markdown is the source of truth; the bridge does not pre-render assistant Markdown to HTML.
+- Before native Rich Markdown delivery, the bridge normalizes known Bot-API-fragile source forms without changing visible meaning, including space-after-marker blockquotes and dollar-prefixed ticker atoms that Telegram may otherwise treat as unterminated math.
+- Quoted rich replies use Telegram `rich_message` blocks as the prompt-context source when available, so `[reply]` context receives rendered plain text instead of raw `InputRichMessage.markdown` fallback text.
 - Long native Markdown replies are split only at Telegram Rich Message transport limits.
 - Streaming previews pass assistant Markdown through to `sendRichMessageDraft` with ownership checks, voice suppression, serialized flushes, and an editable plain-message fallback when draft delivery is unavailable.
 - Preview flushes are serialized so older edits cannot race newer drafts; final delivery waits for active draft flushes and does not perform a post-final draft-clear call.

lib/media.ts

@@ -29,10 +29,15 @@ export interface TelegramVoice {
   mime_type?: string;
 }
 
+export interface TelegramRichMessage {
+  blocks?: unknown[];
+}
+
 export interface TelegramReplyToMessage {
   message_id?: number;
   text?: string;
   caption?: string;
+  rich_message?: TelegramRichMessage;
 }
 
 export interface TelegramSticker {
@@ -43,6 +48,7 @@ export interface TelegramMediaMessage {
   message_id: number;
   text?: string;
   caption?: string;
+  rich_message?: TelegramRichMessage;
   reply_to_message?: TelegramReplyToMessage;
   media_group_id?: string;
   photo?: TelegramPhotoSize[];
@@ -173,10 +179,94 @@ function isImageMimeType(mimeType: string | undefined): boolean {
   return mimeType?.toLowerCase().startsWith("image/") ?? false;
 }
 
+function getObjectField(value: unknown, field: string): unknown {
+  if (typeof value !== "object" || value === null || !(field in value)) {
+    return undefined;
+  }
+  return Reflect.get(value, field);
+}
+
+function joinRichTextParts(parts: string[], separator = ""): string {
+  return parts.filter(Boolean).join(separator).trim();
+}
+
+function extractTelegramRichText(value: unknown): string {
+  if (typeof value === "string") return value;
+  if (Array.isArray(value)) {
+    return joinRichTextParts(value.map(extractTelegramRichText));
+  }
+  if (typeof value !== "object" || value === null) return "";
+  const text = getObjectField(value, "text");
+  if (text !== undefined) return extractTelegramRichText(text);
+  const expression = getObjectField(value, "expression");
+  if (typeof expression === "string") return expression;
+  const alternativeText = getObjectField(value, "alternative_text");
+  if (typeof alternativeText === "string") return alternativeText;
+  return "";
+}
+
+function extractTelegramRichBlockText(block: unknown): string {
+  if (typeof block !== "object" || block === null) return "";
+  const directText = extractTelegramRichText(getObjectField(block, "text"));
+  if (directText) return directText;
+  const summary = extractTelegramRichText(getObjectField(block, "summary"));
+  const nestedBlocks = extractTelegramRichMessageBlocksText(
+    getObjectField(block, "blocks"),
+  );
+  const items = getObjectField(block, "items");
+  const itemText = Array.isArray(items)
+    ? items
+        .map((item) => {
+          const label = getObjectField(item, "label");
+          const body = extractTelegramRichMessageBlocksText(
+            getObjectField(item, "blocks"),
+          );
+          return typeof label === "string" && body ? `${label} ${body}` : body;
+        })
+        .filter(Boolean)
+        .join("\n")
+    : "";
+  const cells = getObjectField(block, "cells");
+  const cellText = Array.isArray(cells)
+    ? cells
+        .map((row) =>
+          Array.isArray(row)
+            ? row
+                .map((cell) => extractTelegramRichText(getObjectField(cell, "text")))
+                .filter(Boolean)
+                .join(" | ")
+            : "",
+        )
+        .filter(Boolean)
+        .join("\n")
+    : "";
+  const caption = extractTelegramRichText(getObjectField(block, "caption"));
+  return joinRichTextParts(
+    [summary, nestedBlocks, itemText, cellText, caption],
+    "\n",
+  );
+}
+
+function extractTelegramRichMessageBlocksText(blocks: unknown): string {
+  if (!Array.isArray(blocks)) return "";
+  return joinRichTextParts(blocks.map(extractTelegramRichBlockText), "\n\n");
+}
+
+function extractTelegramRichMessageText(
+  richMessage: TelegramRichMessage | undefined,
+): string {
+  return extractTelegramRichMessageBlocksText(richMessage?.blocks);
+}
+
 export function extractTelegramMessageText(
   message: TelegramMediaMessage,
 ): string {
-  return (message.text || message.caption || "").trim();
+  return (
+    extractTelegramRichMessageText(message.rich_message) ||
+    message.text ||
+    message.caption ||
+    ""
+  ).trim();
 }
 
 function truncateTelegramReplyContextText(text: string): string {
@@ -188,6 +278,7 @@ export function extractTelegramReplyContextText(
   message: TelegramMediaMessage,
 ): string {
   const quoted = (
+    extractTelegramRichMessageText(message.reply_to_message?.rich_message) ||
     message.reply_to_message?.text ||
     message.reply_to_message?.caption ||
     ""

lib/preview.ts

@@ -10,7 +10,10 @@ import type {
   TelegramSendMessageBody,
   TelegramSentMessage,
 } from "./telegram-api.ts";
-import { buildTelegramReplyParameters } from "./replies.ts";
+import {
+  buildTelegramReplyParameters,
+  normalizeTelegramNativeMarkdown,
+} from "./replies.ts";
 import { stripTelegramCommentMarkupForPreview } from "./outbound.ts";
 import { shouldSuppressPreviewForVoice } from "./voice.ts";
 
@@ -298,7 +301,10 @@ export function createTelegramNativeMarkdownMessageEditor<TReplyMarkup>(deps: {
     deps.editMessageText({
       chat_id: chatId,
       message_id: messageId,
-      rich_message: { markdown, skip_entity_detection: true },
+      rich_message: {
+        markdown: normalizeTelegramNativeMarkdown(markdown),
+        skip_entity_detection: true,
+      },
       ...(options?.replyMarkup ? { reply_markup: options.replyMarkup } : {}),
     });
 }
@@ -632,7 +638,11 @@ async function performTelegramPreviewFlush<
     const draftId = state.draftId ?? deps.allocateDraftId();
     state.draftId = draftId;
     try {
-      await deps.sendDraft(chatId, draftId, snapshot.sourceText);
+      await deps.sendDraft(
+        chatId,
+        draftId,
+        normalizeTelegramNativeMarkdown(snapshot.sourceText),
+      );
       deps.setDraftSupport("supported");
       state.mode = "draft";
       state.lastSentText = snapshot.text;

lib/prompts.ts

@@ -30,7 +30,7 @@ Telegram-originated turn context:
 Telegram-visible output:
 - Telegram is mobile-first: keep answers easy to scan, use headings/lists when useful, and avoid unnecessarily huge blocks of text.
 - For formulas, use math delimiters like \`$E = mc^2$\` for inline formulas and \`$$\\nE = mc^2\\n$$\` for block formulas; do not wrap formulas in backticks unless they should render as literal code.
-- Wide monospace blocks can become unreadable on mobile; use them only when structure or literal code requires them.
+- Use inline code for short copyable literals (commands, paths, IDs, symbols); avoid wide monospace blocks unless structure or literal code requires them.
 - For requested/generated files, call \`telegram_attach(local_path)\`; during Telegram turns it attaches files to the active reply, and during explicit local/TUI Telegram-delivery requests it sends files directly to the paired/default chat or an explicit \`chat_id\`. If a local/TUI user explicitly asks to send a text message to Telegram, use \`telegram_message\` with Markdown text; embed the same top-level \`telegram_button\` comments when inline prompt buttons are needed, because Telegram buttons must belong to a message. Direct local/TUI Telegram delivery requires this π instance to own \`/telegram-connect\`; if ownership is elsewhere, connect/take over first instead of bypassing the lock.
 
 Native outbound actions:

lib/replies.ts

@@ -257,17 +257,58 @@ export async function sendTelegramPlainReply(
   return deps.sendRenderedChunks(chunks);
 }
 
+function normalizeTelegramNativeMarkdownLine(line: string): string {
+  let result = line.replace(/^( {0,3}>)[ \t]/, "$1");
+  const codeSpans: string[] = [];
+  result = result.replace(/`+[^`]*`+/g, (code) => {
+    const token = `\u0000${codeSpans.length}\u0000`;
+    codeSpans.push(code);
+    return token;
+  });
+  result = result.replace(
+    /(^|[^\\$])\$([A-Z][A-Z0-9]{1,})(?!\$)(?=\b|[.,;:)/-])/g,
+    (_match, prefix: string, ticker: string) => `${prefix}\\$${ticker}`,
+  );
+  return result.replace(/\u0000(\d+)\u0000/g, (_match, index) => codeSpans[Number(index)] ?? "");
+}
+
+export function normalizeTelegramNativeMarkdown(markdown: string): string {
+  const lines = markdown.replace(/\r\n/g, "\n").split("\n");
+  let fence: { marker: "`" | "~"; length: number } | undefined;
+  return lines
+    .map((line) => {
+      const fenceMatch = line.match(/^ {0,3}(`{3,}|~{3,})/);
+      const inFence = fence !== undefined;
+      if (!inFence && fenceMatch) {
+        const markerText = fenceMatch[1] ?? "```";
+        fence = { marker: markerText[0] as "`" | "~", length: markerText.length };
+        return line;
+      }
+      if (
+        inFence &&
+        new RegExp(`^ {0,3}${fence?.marker}{${fence?.length},}\\s*$`).test(line)
+      ) {
+        fence = undefined;
+        return line;
+      }
+      if (!inFence) return normalizeTelegramNativeMarkdownLine(line);
+      return line;
+    })
+    .join("\n");
+}
+
 export function splitTelegramNativeMarkdown(markdown: string): string[] {
+  const normalizedMarkdown = normalizeTelegramNativeMarkdown(markdown);
   if (
-    markdown.length <= TELEGRAM_RICH_MESSAGE_MAX_CHARS &&
-    countTelegramNativeMarkdownBlocks(markdown) <= TELEGRAM_RICH_MESSAGE_MAX_BLOCKS
+    normalizedMarkdown.length <= TELEGRAM_RICH_MESSAGE_MAX_CHARS &&
+    countTelegramNativeMarkdownBlocks(normalizedMarkdown) <= TELEGRAM_RICH_MESSAGE_MAX_BLOCKS
   ) {
-    return [markdown];
+    return [normalizedMarkdown];
   }
   const chunks: string[] = [];
   let current = "";
   let currentBlockCount = 0;
-  for (const rawBlock of splitTelegramNativeMarkdownBlocks(markdown)) {
+  for (const rawBlock of splitTelegramNativeMarkdownBlocks(normalizedMarkdown)) {
     for (const block of splitTelegramNativeMarkdownCountedBlocks(rawBlock)) {
       const blockCount = countTelegramNativeMarkdownBlocks(block);
       const candidate = current ? `${current}\n\n${block}` : block;

lib/telegram-api.ts

@@ -101,12 +101,18 @@ export interface TelegramSticker {
   emoji?: string;
 }
 
+export interface TelegramRichMessage {
+  blocks?: unknown[];
+  is_rtl?: boolean;
+}
+
 export interface TelegramMessage {
   message_id: number;
   chat: TelegramChat;
   from?: TelegramUser;
   text?: string;
   caption?: string;
+  rich_message?: TelegramRichMessage;
   media_group_id?: string;
   photo?: TelegramPhotoSize[];
   document?: TelegramDocument;
@@ -160,6 +166,7 @@ export interface TelegramGuestMessage {
   date: number;
   text?: string;
   caption?: string;
+  rich_message?: TelegramRichMessage;
   guest_query_id: string;
   guest_bot_caller_user?: TelegramUser;
   guest_bot_caller_chat?: TelegramChat;

package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "@llblab/pi-telegram",
-  "version": "0.17.0",
+  "version": "0.17.1",
   "private": false,
   "publishConfig": {
     "access": "public"