Claude Code transcript noise: <local-command-*> tags and ANSI escapes survive strip_noise()

[lucagentile]

What happened?

After mining a Claude Code session into the palace, drawers contain raw <local-command-stdout> envelopes, bare <command-args></command-args> remnants, and ANSI escape sequences from Bash tool output. These pollute search results, bloat embeddings (each escape is several BPE tokens), and produce drawers that are mostly noise.

Real samples from a normal mining run (ESC denotes the literal \x1b byte stored in drawer content):

> <local-command-stdout>Set defaultPermissionMode to ESC[1mdefaultESC[22m
Disabled auto-compact Disabled Remote Control for all sessions</local-command-stdout>

> <command-message>clear</command-message>
<command-args></command-args>

ESC[38;2;153;153;153m├ESC[39m mempalace_add_drawer ESC[38;2;153;153;153m├ESC[39m mempalace_check_duplicate ...

In one palace I inspected, ~85% of drawers in wing=sessions landed in room=technical and many were dominated by ANSI bytes from /context / /help output, or were the empty <command-args></command-args> remnant of slash-command invocations.

What did you expect?

The existing strip_noise() machinery in mempalace/normalize.py already strips <system-reminder>, <command-message>, and <command-name>. The same hygiene should apply to:

1. The rest of the slash-command envelope Claude Code injects (<local-command-caveat>, <command-args>, <local-command-stdout>).
2. ANSI control sequences (CSI + OSC) that Claude Code's Bash tool faithfully preserves from terminal output.

How to reproduce:

1. Run any Claude Code session that uses slash commands (/clear, /effort, /help, plugin commands) and any Bash tool calls producing colored output (most modern CLIs).
2. Wait for the mempalace Stop hook to mine the transcript, or run manually:

   mempalace mine ~/.claude/projects/<your-folder> --mode convos --wing sessions
   

3. Inspect the result, e.g.:

   mempalace_list_drawers wing=sessions
   

   Drawers will contain ANSI-laden content and surviving <local-command-*> tag envelopes.

Environment:

• OS: macOS 25.3.0 (Darwin)
• Python: 3.14
• MemPalace: 3.3.3 (verified develop tip has identical relevant code path in mempalace/normalize.py)
• Harness: Claude Code

Root-cause sketch

Claude Code wraps every slash-command invocation in a five-tag envelope:

<local-command-caveat>...</local-command-caveat>
<command-name>...</command-name>
<command-message>...</command-message>
<command-args>...</command-args>
<local-command-stdout>...</local-command-stdout>

_NOISE_TAGS in mempalace/normalize.py covers command-name and command-message; the other three slip through. Separately, Bash tool_result content is captured verbatim — including ANSI bytes — and strip_noise() does not currently sweep them.

Fix: extend _NOISE_TAGS with command-args, local-command-caveat, local-command-stdout, and add CSI + OSC ANSI strippers applied after tag removal in strip_noise(). Each ANSI pattern is anchored on the literal ESC byte so user prose that mentions e.g. [1m] by name stays intact. PR coming separately.

[Necmttn]

I would make this a structured Claude-transcript normalization step before embedding, not only a regex cleanup pass.

Keep the raw transcript ref, then materialize a cleaned text projection that drops local-command envelopes, empty command args, ANSI CSI/OSC sequences, and slash-command scaffolding. Tool results can still be summarized or stored as tool metadata when useful.

That prevents noisy drawers while preserving a route back to the original line when the cleaner is wrong.

---

Generated with ax.

[KeilerHirsch]

Opened #1909 to close this.

One addition worth flagging: the root-cause sketch above lists a five-tag envelope, but enumerating every <(local-)?command-*> tag across a live ~/.claude/projects corpus turns up a sixth — <local-command-stderr> — emitted for slash commands that write to stderr. It leaks the same way, so #1909 strips all six (command-args, local-command-caveat, local-command-stdout, local-command-stderr) plus the ANSI CSI/OSC escapes, and bumps NORMALIZE_VERSION 2 → 3 so already-mined drawers rebuild.

#1909 supersedes the two earlier attempts and folds in their review feedback: #1334 was flagged for not bumping NORMALIZE_VERSION (done), and #1701 is CONFLICTING/stale and also misses the stderr tag. It rebases clean on develop with 12 new tests.