/run and /test read command output one character at a time (~60x slower than bulk reads)

[aniruddhaadak80]

Issue

Summary

run_cmd_subprocess() — the engine behind /run, /test, --auto-test and model-suggested shell commands on non-tty/Windows paths — reads the child process's output one character at a time:

aider/run_cmd.py:

process = subprocess.Popen(
    command,
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT,
    text=True,
    shell=True,
    encoding=encoding,
    errors="replace",
    bufsize=0,  # Set bufsize to 0 for unbuffered output
    universal_newlines=True,
    cwd=cwd,
)

output = []
while True:
    chunk = process.stdout.read(1)      # <-- one character per iteration
    if not chunk:
        break
    print(chunk, end="", flush=True)    # <-- and one flush per character
    output.append(chunk)

Two compounding problems:

1. read(1) in a Python loop — with text=True, each call goes through the TextIOWrapper decode layer; 2 MB of output means ~2,000,000 iterations plus 2,000,000 print(..., flush=True) calls.
2. bufsize=0 with universal_newlines=True — the comment says unbuffered for real-time display, but the right tool for that is a modest read size (e.g. readline() or read(4096)), not zero buffering + single-char reads.

Measured impact (Windows, Python 3.12, current main)

Consuming 2 MB of output from a trivial subprocess:

aider run_cmd_subprocess (read(1) loop):   47.49s
baseline subprocess bulk capture:           0.79s

That's ~60x slower than necessary, pure interpreter overhead. Any /run of a chatty build or a verbose test suite stalls aider for tens of seconds after the child has finished producing data; under --auto-test --test-cmd ... headless runs this delay is incurred automatically on every failing round. The same loop also runs for /test (cmd_run(add_on_nonzero_exit=True)).

Suggested fix

Keep the real-time echo but read chunks instead of characters:

while True:
    chunk = process.stdout.read(4096)
    if not chunk:
        break
    print(chunk, end="", flush=True)
    output.append(chunk)

or iterate for line in iter(process.stdout.readline, ""):. Either preserves incremental display while reducing the loop count by orders of magnitude. I measured locally: switching to read(4096) brings the same 2 MB case from ~47s down to parity with the baseline (<1s). Happy to send this as a PR.

Version and model info

• Aider v0.86.3.dev53+g5dc9490bb (main @ 5dc9490bb)
• Python 3.12.10, Windows 11 (the read(1) path is cross-platform: it is what non-interactive shells and CI fall back to via sys.stdin.isatty() check at run_cmd())
• Affects /run, /test, --auto-test, and suggested-command execution whenever pexpect isn't used

[lx3133584]

I have submitted a fix for this issue in #5625 with reproduction tests.