Skip to content

[Bug] Agent.run_batched raises before running anything when imgs is omitted, and mislabels images when it is not - #1938

Merged
kyegomez merged 1 commit into
kyegomez:masterfrom
ayaangazali:fix/run-batched-imgs-shadow
Aug 20, 2026
Merged

[Bug] Agent.run_batched raises before running anything when imgs is omitted, and mislabels images when it is not#1938
kyegomez merged 1 commit into
kyegomez:masterfrom
ayaangazali:fix/run-batched-imgs-shadow

Conversation

@ayaangazali

Copy link
Copy Markdown
Contributor

Problem

Agent.run_batched uses imgs as both the parameter and the loop variable:

return [
    self.run(task=task, imgs=imgs, *args, **kwargs)
    for task, imgs in zip(tasks, imgs)
]

Three separate failures come out of those four lines:

1. The documented default call raises. imgs defaults to None and the docstring says it is optional, but zip(tasks, None) fails before any task runs:

>>> agent.run_batched(["t1", "t2"])
TypeError: 'NoneType' object is not iterable

2. With images, each task gets a str in a List[str] field. The loop variable rebinds the parameter, so self.run receives one image path in imgs, which Agent.runcall_llmllm_manager.call forwards to the provider as run_args["imgs"] = "a.png". The parameter for a single image is img.

>>> agent.run_batched(["t1", "t2"], imgs=["a.png", "b.png"])
[{'task': 't1', 'imgs': 'a.png'}, {'task': 't2', 'imgs': 'b.png'}]
                        ^^^^ a list field holding one string

3. Unequal lengths silently drop tasks. zip stops at the shorter sequence, so three tasks and one image runs one task and discards two, with no error and no log line.

Fix

if imgs is None:
    return [self.run(task=task, *args, **kwargs) for task in tasks]

if len(imgs) != len(tasks):
    raise ValueError(...)

return [
    self.run(task=task, img=img, *args, **kwargs)
    for task, img in zip(tasks, imgs)
]

After:

no imgs        -> [{'task': 't1'}, {'task': 't2'}]
paired imgs    -> [{'task': 't1', 'img': 'a.png'}, {'task': 't2', 'img': 'b.png'}]
length mismatch-> ValueError: run_batched got 3 tasks and 1 images; pass one image
                  per task, or omit imgs entirely. Zipping them would silently drop the extras.

The docstring said "Run a batch of tasks concurrently" while the body has always been a plain list comprehension, so it now says what it does. Making it genuinely concurrent is a behaviour change — the sibling paths use a call-scoped ContextThreadPoolExecutor after #1909 — and belongs in its own PR rather than smuggled into a crash fix.

Tests

Three cases added to tests/structs/test_agent.py, next to TestConcurrentExecutionPool, which is where the other Agent batch-path tests live. They build the agent with Agent.__new__ like that class does, so there is no model client, no memory file and no provider call.

All three fail on master with the source reverted and the tests kept:

FAILED TestRunBatchedImagePairing::test_tasks_without_images_run
FAILED TestRunBatchedImagePairing::test_each_task_gets_its_own_image_as_a_single_image
FAILED TestRunBatchedImagePairing::test_mismatched_lengths_raise_instead_of_dropping_tasks
E   Failed: DID NOT RAISE ValueError

tests/structs/test_agent.py goes from 19 failed, 71 passed, 8 errors on master to 19 failed, 74 passed, 8 errors here — same failure set, plus the three new tests. Those 19 are pre-existing and need provider credentials.

🤖 Generated with Claude Code

`run_batched` zips the tasks against `imgs` using `imgs` as the loop variable:

    return [
        self.run(task=task, imgs=imgs, *args, **kwargs)
        for task, imgs in zip(tasks, imgs)
    ]

Three problems in four lines:

1. `imgs` defaults to None and is documented as optional, but `zip(tasks, None)`
   raises `TypeError: 'NoneType' object is not iterable` — so the documented
   basic call `agent.run_batched(["a", "b"])` never runs a single task.
2. The loop variable rebinds the parameter, so each `self.run` call receives
   one image string in `imgs`, a field declared `List[str]` and passed straight
   through to the provider call. The single-image parameter is `img`.
3. Unequal lengths zip to the shorter one, so passing fewer images than tasks
   silently discards tasks rather than reporting the mismatch.

Now: no images runs the tasks plainly, paired images go through `img`, and a
length mismatch raises rather than dropping work. The docstring said
"concurrently" while the body was always a list comprehension; it now says
what it does. Making it actually concurrent is a behaviour change and belongs
in its own PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ayaangazali
ayaangazali requested a review from kyegomez as a code owner August 20, 2026 16:25
Copilot AI lite review requested due to automatic review settings August 20, 2026 16:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kyegomez
kyegomez merged commit ac0f6e8 into kyegomez:master Aug 20, 2026
5 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants