Skip to content

SPEC_MUNCH.md

Latest commit

 

History

History
252 lines (183 loc) · 8.24 KB

SPEC_MUNCH.md

File metadata and controls

252 lines (183 loc) · 8.24 KB
Raw

MUNCH — Compact Output Encoding Specification

Version: 1 Status: stable (since jcodemunch-mcp 1.54.0)

MUNCH is jcodemunch's purpose-built wire format for MCP tool responses. It is an opt-in second-axis token optimization that complements the retrieval-side savings jcodemunch already provides: retrieval decides what to send; MUNCH decides how to pack it.

This document specifies the on-wire format so third-party clients and alternate MCP servers can decode MUNCH payloads without depending on the Python reference decoder.

1. Design goals

  1. Deterministic — the same response dict encodes to the same bytes.
  2. Round-trippable — a decoder reconstructs the original dict structure (top-level keys, column order, per-column types) for all schemas covered by a tier-1 encoder, and for most shapes handled by the generic fallback.
  3. Streaming-friendly — the format is line-oriented; a decoder can tokenise sections with splitlines() and a blank-line separator.
  4. Fail-open — any client that cannot decode MUNCH can request format="json" and receive a standard JSON dict.
  5. Size-justified — encoding only fires when it saves at least 15% of bytes (configurable). Small payloads fall back to JSON automatically.

2. Envelope

A MUNCH payload is a UTF-8 text document with one header line followed by one or more sections separated by a single blank line (\n\n). Sections themselves may contain multiple \n-terminated lines.

#MUNCH/<VERSION> tool=<tool_name> enc=<encoding_id>

<optional legend section>

<scalars section>

<optional table section 1>
<optional table section 2>
...

All payloads end with a trailing \n.

2.1 Header

Exactly one line, always the first line of the payload. Format:

#MUNCH/1 tool=<tool_name> enc=<encoding_id>
  • VERSION is an integer. This spec describes version 1.
  • tool_name is the MCP tool name (e.g. find_references).
  • encoding_id is a short mnemonic identifying the encoder/schema used (e.g. gen1, fr1, dg1). Decoders select the matching rehydration routine by this id.

Decoders must reject any payload whose first line does not start with #MUNCH/.

3. Legends section

Optional. Interns frequently-repeated string prefixes (typically file paths) so each subsequent occurrence collapses to a short numeric handle.

Format — one line per entry, each:

@<N>=<literal prefix string>
  • @ is the fixed legend prefix.
  • N is a 1-based integer handle. Handles are dense (@1, @2, ...).
  • The literal is an unquoted string running to end-of-line. It may contain any character except \n.

Values that match a registered prefix are rewritten in the output as @N<remainder>. For example, given @1=src/models/, the path src/models/user.py is emitted as @1user.py. Decoding reverses this: any token starting with @ followed by digits has those digits parsed as a handle, the literal substituted, and the trailing remainder kept.

Tokens that start with @ but do not match a known handle are kept verbatim.

4. Scalars section

Exactly one line of whitespace-separated key=value pairs. Keys are MCP response field names. Values follow these rules:

  • Strings — emitted as-is if they contain none of , = <whitespace> ". Otherwise double-quoted with doubled-quote escaping ("a ""b"" c" represents a "b" c), per RFC 4180.
  • Integers — decimal, no quotes (42).
  • Floats — Python repr(float) form (0.15, 1.5e-05).
  • Booleans — T (true) or F (false).
  • Null — empty string. Empty strings are encoded as "" to disambiguate.
  • Nested dicts and complex objects — flattened via a dotted key (symbol.file=x.py) or wrapped as a JSON blob scalar (__json.channels={"a":1}). A dedicated __json. prefix signals to the decoder that the value is a JSON string to rehydrate.

4.1 Reserved scalar keys

Generic-encoder payloads include these synthetic scalars (tier-1 encoders carry the same information in a schema file instead):

  • __tables — comma-separated table schema. One entry per table: <tag>:<original_key>:<col1>|<col2>|...:<type1>|<type2>|.... Types are int, float, bool, or str. Example: t:references:file|line|kind:str|int|str.
  • __stypes — pipe-separated scalar type map for any non-string top-level scalar. Example: total:int|ratio:float|ok:bool.

Decoders must pop these keys before materialising the response.

4.2 Meta fields

_meta values are flattened with the _meta. prefix (_meta.timing_ms=3.1 _meta.truncated=F). Decoders re-nest them into a single _meta dict on output.

5. Tables section

Each table is a separate section. A section is one or more CSV-style rows where the first field is a single-character tag. Rows are emitted with RFC 4180 quoting. The first field discriminates both which table a row belongs to and, when multiple tables share a section block, how to re-split them.

Example:

t,src/a.py,10,call
t,src/a.py,22,call
t,src/b.py,5,ref

With schema t:references:file|line|kind:str|int|str, this decodes to:

{
  "references": [
    {"file": "src/a.py", "line": 10, "kind": "call"},
    {"file": "src/a.py", "line": 22, "kind": "call"},
    {"file": "src/b.py", "line": 5,  "kind": "ref"}
  ]
}

5.1 Tag alphabet

Tags are single characters drawn from a–z. The first non-header section whose first line matches ^[a-z], is classified as a table section; any other section is treated as the scalars section (or, if it begins with @, the legend section).

5.2 Per-column interning

Columns declared in a tier-1 encoder's intern set are legend-prefix-encoded using the shared legend. The generic encoder interns any string column whose prefix repeats enough times to save bytes.

5.3 Type coercion

On decode, each field is coerced using the column's declared type:

  • str — legend-decoded then returned as-is.
  • int — int(raw), unchanged on ValueError.
  • float — float(raw), unchanged on ValueError.
  • bool — T → True, anything else → False.
  • Empty field ("") always decodes to None.

6. Dispatcher contract

Every MCP tool in jcodemunch accepts an optional format argument:

  • "auto" (default) — encode, measure, emit compact if savings ≥ threshold, otherwise emit JSON. Threshold is 15% by default, overridable via JCODEMUNCH_ENCODING_THRESHOLD.
  • "compact" — always emit MUNCH (no threshold check).
  • "json" — never encode; emit standard JSON.

Server-wide default is overridable via JCODEMUNCH_DEFAULT_FORMAT.

Every compact response adds three fields to _meta:

  • encoding — the encoding id used (e.g. fr1, gen1).
  • encoding_tokens_saved — estimated tokens saved for this call.
  • total_encoding_tokens_saved — lifetime total across the session.

Clients that cannot decode MUNCH should request format="json".

7. Reference decoder

The canonical decoder ships at jcodemunch_mcp.encoding.decoder.decode. It:

  1. Returns json.loads(payload) if the input does not start with #MUNCH/.
  2. Parses the header, resolves the encoder module by enc= id.
  3. Delegates to that module's decode(payload) function.

Tier-1 encoder ids currently defined (v1.56.0):

id tool
fr1 find_references
fi1 find_importers
ch1 get_call_hierarchy
dg1 get_dependency_graph
br1 get_blast_radius
ip1 get_impact_preview
sc1 get_signal_chains
dc1 get_dependency_cycles
tm1 get_tectonic_map
ss1 search_symbols
st1 search_text
sa1 search_ast
fo1 get_file_outline
ro1 get_repo_outline
rc1 get_ranked_context
gen1 generic fallback for all other tools

Any tool not listed above still encodes through gen1, which is round-trippable for homogeneous-table responses and falls open to JSON passthrough when the savings gate rejects the result.

8. Versioning

Changes that would break an existing decoder (re-purposing a character, changing the scalar delimiter, etc.) bump the major version in the header (#MUNCH/2). Additive changes (new encoding ids, new reserved scalar keys) keep version 1 and new clients must ignore unknown scalar keys they do not recognise.