BUG: RepoMap.get_tags_raw loses or duplicates Tree-sitter captures #5580

Issue

Summary

In aider/repomap.py, RepoMap.get_tags_raw does not preserve the nodes returned for each Tree-sitter capture.

When USING_TSL_PACK is false, only the final node for each capture tag is retained. When it is true, every node is retained but the final node is appended a second time.

For two captured definitions named alpha and beta, the two branches therefore produce:

{
    False: [("beta", 2)],
    True: [("alpha", 1), ("beta", 2), ("beta", 2)],
}

Steps to reproduce

1. Check out Aider main at commit 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

2. Add the following test inside TestRepoMap in tests/basic/test_repomap.py:

def test_get_tags_raw_preserves_all_capture_nodes(self):
    from unittest import mock

    from aider import repomap as repomap_module

    class Node:
        def __init__(self, name, line):
            self.text = name.encode()
            self.start_point = (line, 0)

    class IO:
        def read_text(self, _fname):
            return "source"

    nodes = [Node("alpha", 1), Node("beta", 2)]
    captures = {"name.definition.function": nodes}
    scm = mock.Mock()
    scm.exists.return_value = True
    scm.read_text.return_value = "query"
    parser = mock.Mock()
    parser.parse.return_value = mock.Mock(root_node=object())
    repo_map = RepoMap.__new__(RepoMap)
    repo_map.io = IO()
    repo_map._run_captures = mock.Mock(return_value=captures)
    observed = {}

    for using_tsl_pack in (False, True):
        with mock.patch.multiple(
            repomap_module,
            filename_to_lang=mock.Mock(return_value="python"),
            get_language=mock.Mock(return_value=object()),
            get_parser=mock.Mock(return_value=parser),
            get_scm_fname=mock.Mock(return_value=scm),
            Query=mock.Mock(return_value=object()),
            USING_TSL_PACK=using_tsl_pack,
        ):
            tags = list(repo_map.get_tags_raw("sample.py", "sample.py"))
            observed[using_tsl_pack] = [
                (tag.name, tag.line) for tag in tags if tag.kind == "def"
            ]

    expected = [("alpha", 1), ("beta", 2)]
    self.assertEqual(observed, {False: expected, True: expected})

3. Run:

python -m pytest tests/basic/test_repomap.py -q

4. Observe that the newly added assertion fails while the 45 existing tests pass.

Expected behavior

Each node returned for a Tree-sitter capture should produce exactly one corresponding tag, independently of the Tree-sitter compatibility branch:

{
    False: [("alpha", 1), ("beta", 2)],
    True: [("alpha", 1), ("beta", 2)],
}

Actual behavior

The non-TSL-pack branch loses the first captured node. The TSL-pack branch duplicates the final captured node:

{
    False: [("beta", 2)],
    True: [("alpha", 1), ("beta", 2), ("beta", 2)],
}

The focused run reports:

....F.........................................                           [100%]

FAILED tests/basic/test_repomap.py::TestRepoMap::test_get_tags_raw_preserves_all_capture_nodes

AssertionError: {False: [('beta', 2)], True: [('alpha', 1), ('beta', 2), ('beta', 2)]} != {False: [('alpha', 1), ('beta', 2)], True: [('alpha', 1), ('beta', 2)]}

- {False: [('beta', 2)], True: [('alpha', 1), ('beta', 2), ('beta', 2)]}
+ {False: [('alpha', 1), ('beta', 2)], True: [('alpha', 1), ('beta', 2)]}

1 failed, 45 passed in 19.86s

Impact

get_tags_raw supplies the definition and reference tags used to construct repository maps.

Depending on the active Tree-sitter compatibility path, its output can omit captured definitions or references, or contain a duplicate of the final capture. The tag stream consumed by repository-map ranking therefore differs from the capture results returned by Tree-sitter.

Root cause

The capture loop currently appends each node to captures_by_tag, then appends the final loop variable once more outside the inner loop. Only that final node is added to matches:

captures_by_tag = defaultdict(list)
matches = []
for tag, nodes in captures.items():
    for node in nodes:
        captures_by_tag[tag].append(node)
    captures_by_tag[tag].append(node)
    matches.append((node, tag))

The TSL-pack branch builds all_nodes from captures_by_tag, so it receives the duplicated final node. The compatibility branch builds all_nodes from matches, so it receives only the final node for each tag.

Possible fix direction

Record both collections once per captured node:

captures_by_tag = defaultdict(list)
matches = []
for tag, nodes in captures.items():
    for node in nodes:
        captures_by_tag[tag].append(node)
        matches.append((node, tag))

Regression coverage should exercise both values of USING_TSL_PACK and verify that each captured node produces exactly one tag.

Version and model info

Aider: 0.86.3.dev, main@5dc9490bb35f9729ef2c95d00a19ccd30c26339c
Python: 3.12.13
Operating system: macOS 15.7.3
Installation: source checkout; focused pytest reproduction
Model: local gpt-3.5-turbo model metadata initialized by TestRepoMap; no provider request is made
Configuration: both USING_TSL_PACK=False and USING_TSL_PACK=True compatibility branches are exercised directly