Skip to content

Commit 8929e8c

Browse files
authored
Revise LANGUAGE_SUPPORT.md for clarity and updates
Updated the language support documentation to include additional notes and clarifications for each supported language. Revised the structure and formatting for consistency and clarity.
1 parent 0c3bc9c commit 8929e8c

1 file changed

Lines changed: 43 additions & 33 deletions

File tree

LANGUAGE_SUPPORT.md

Lines changed: 43 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -2,79 +2,89 @@
22

33
## Supported Languages
44

5-
| Language | Extensions | Parser | Symbol Types | Decorators | Docstrings | Known Limitations |
6-
|----------|-----------|--------|-------------|-----------|-----------|-------------------|
7-
| Python | `.py` | tree-sitter-python | function, class, method, constant, type | `@decorator` | Triple-quoted strings | Type alias requires Python 3.12+ syntax |
8-
| JavaScript | `.js`, `.jsx` | tree-sitter-javascript | function, class, method, constant | None | `//` / `/** */` comments | Arrow functions without names skipped |
9-
| TypeScript | `.ts`, `.tsx` | tree-sitter-typescript | function, class, method, constant, type | `@decorator` | `//` / `/** */` comments | Decorators require TC39 stage 3+ |
10-
| Go | `.go` | tree-sitter-go | function, method, type, constant | None | `//` comments | No class-like nesting |
11-
| Rust | `.rs` | tree-sitter-rust | function, type (struct/enum/trait), class (impl), constant | `#[attr]` | `///` / `//!` comments | Macro-generated items invisible |
12-
| Java | `.java` | tree-sitter-java | method, class, type (interface/enum), constant | `@Annotation` | `/** */` Javadoc | Inner classes have limited nesting |
5+
| Language | Extensions | Parser | Symbol Types | Decorators | Docstrings | Notes / Limitations |
6+
| ---------- | ------------- | ---------------------- | -------------------------------------------------- | ------------- | -------------------------- | ---------------------------------------------------------------- |
7+
| Python | `.py` | tree-sitter-python | function, class, method, constant, type | `@decorator` | Triple-quoted strings | Type aliases require Python 3.12+ syntax for full fidelity |
8+
| JavaScript | `.js`, `.jsx` | tree-sitter-javascript | function, class, method, constant || `//` and `/** */` comments | Anonymous arrow functions without assigned names are not indexed |
9+
| TypeScript | `.ts`, `.tsx` | tree-sitter-typescript | function, class, method, constant, type | `@decorator` | `//` and `/** */` comments | Decorator extraction depends on Stage-3 decorator syntax |
10+
| Go | `.go` | tree-sitter-go | function, method, type, constant || `//` comments | No class hierarchy (language limitation) |
11+
| Rust | `.rs` | tree-sitter-rust | function, type (struct/enum/trait), impl, constant | `#[attr]` | `///` and `//!` comments | Macro-generated symbols are not visible to the parser |
12+
| Java | `.java` | tree-sitter-java | method, class, type (interface/enum), constant | `@Annotation` | `/** */` Javadoc | Deep inner-class nesting may be flattened |
1313

14-
## Parser: tree-sitter
14+
---
1515

16-
All parsing uses [tree-sitter](https://tree-sitter.github.io/) via the `tree-sitter-language-pack` Python package. This provides:
16+
## Parser Engine
1717

18-
- Incremental, error-tolerant parsing
19-
- Consistent AST representation across languages
20-
- Pre-compiled bindings for all 6 languages
18+
All language parsing is powered by **tree-sitter** via the `tree-sitter-language-pack` Python package, providing:
19+
20+
* Incremental, error-tolerant parsing
21+
* Uniform AST representation across languages
22+
* Pre-compiled grammars for supported languages
2123

2224
**Dependency:** `tree-sitter-language-pack>=0.7.0` (pinned in `pyproject.toml`)
2325

26+
---
27+
2428
## Adding a New Language
2529

26-
1. **Create a `LanguageSpec`** in `src/jcodemunch_mcp/parser/languages.py`:
30+
1. **Define a `LanguageSpec`** in `src/jcodemunch_mcp/parser/languages.py`:
2731

2832
```python
2933
NEW_LANG_SPEC = LanguageSpec(
30-
ts_language="new_language", # tree-sitter grammar name
31-
symbol_node_types={ # AST node type -> symbol kind
34+
ts_language="new_language",
35+
symbol_node_types={
3236
"function_definition": "function",
3337
"class_definition": "class",
3438
},
35-
name_fields={ # How to extract names
39+
name_fields={
3640
"function_definition": "name",
3741
"class_definition": "name",
3842
},
39-
param_fields={ # Parameter extraction
43+
param_fields={
4044
"function_definition": "parameters",
4145
},
42-
return_type_fields={}, # Return type extraction
43-
docstring_strategy="preceding_comment", # or "next_sibling_string"
44-
decorator_node_type=None, # Decorator node type if any
45-
container_node_types=["class_definition"], # Nesting containers
46-
constant_patterns=[], # Constant node types
47-
type_patterns=[], # Type definition node types
46+
return_type_fields={},
47+
docstring_strategy="preceding_comment",
48+
decorator_node_type=None,
49+
container_node_types=["class_definition"],
50+
constant_patterns=[],
51+
type_patterns=[],
4852
)
4953
```
5054

51-
2. **Register it** in `LANGUAGE_REGISTRY`:
55+
2. **Register the language**:
56+
5257
```python
5358
LANGUAGE_REGISTRY["new_language"] = NEW_LANG_SPEC
5459
```
5560

56-
3. **Add file extensions** to `LANGUAGE_EXTENSIONS`:
61+
3. **Map file extensions**:
62+
5763
```python
5864
LANGUAGE_EXTENSIONS[".ext"] = "new_language"
5965
```
6066

61-
4. **Verify** the tree-sitter grammar name exists in `tree-sitter-language-pack`:
67+
4. **Verify parser availability**:
68+
6269
```python
6370
from tree_sitter_language_pack import get_parser
64-
parser = get_parser("new_language") # Must not raise
71+
get_parser("new_language") # Must not raise
6572
```
6673

67-
5. **Write tests** in `tests/test_parser.py`:
74+
5. **Add parser tests**:
75+
6876
```python
6977
def test_parse_new_language():
70-
source = "..." # Minimal source with function + class
78+
source = "..."
7179
symbols = parse_file(source, "test.ext", "new_language")
7280
assert len(symbols) >= 2
7381
```
7482

75-
## Debugging AST Node Types
83+
---
84+
85+
## Inspecting AST Node Types
7686

77-
To inspect what tree-sitter produces for a given source file:
87+
To inspect the node types produced by tree-sitter for a source file:
7888

7989
```python
8090
from tree_sitter_language_pack import get_parser
@@ -90,4 +100,4 @@ def print_tree(node, indent=0):
90100
print_tree(tree.root_node)
91101
```
92102

93-
This helps identify the correct `symbol_node_types` and `name_fields` for a new language.
103+
This inspection process helps identify the correct `symbol_node_types`, `name_fields`, and extraction rules when adding support for a new language.

0 commit comments

Comments
 (0)