[BUG] HierarchicalSwarm silently returns no output when given a caller-supplied director

[ayaangazali]

A HierarchicalSwarm built with your own director agent runs to completion, returns successfully, and produces no agent work at all. This is the pattern the project documents:

director = Agent(agent_name="Director", model_name="...", max_loops=1)
swarm = HierarchicalSwarm(director=director, agents=workers, max_loops=1)
swarm.run(task)

Before this patch that returns 61 characters:

[{"role": "System", "content": "--- Loop 1/1 completed ---"}]

No exception, no failed status. The workers never ran.

Minimal reproduction

from swarms import Agent, HierarchicalSwarm
M = "claude-haiku-4-5-20251001"

def mk(n, p):
    return Agent(agent_name=n, agent_description=p, system_prompt=p,
                 model_name=M, max_loops=1, max_tokens=200, persistent_memory=False)

workers = [mk("Writer", "Write concise prose."), mk("Checker", "Fact-check concisely.")]

# A: let the framework build the director
print(HierarchicalSwarm(agents=workers, director_model_name=M, max_loops=1)
      .run("Write two sentences on why Python is popular for AI."))
# -> roles [Writer, Checker, Director, System], ~5000 chars

# B: supply the director yourself (documented pattern)
print(HierarchicalSwarm(director=mk("Director", "Break the task into subtasks and delegate."),
                        agents=workers, max_loops=1)
      .run("Write two sentences on why Python is popular for AI."))
# -> roles [System], 61 chars

Root cause

Two independent defects combine, which is why this stayed hidden.

1. A caller-supplied director never receives the SwarmSpec schema.

reliability_checks() attaches the schema only on the path where it constructs the director itself:

if self.director is None:
    self.director = self.setup_director()   # gets tools_list_dictionary=[SwarmSpec schema]

A director passed in by the caller is stored verbatim during __init__, so it replies in prose and parse_orders raises JSONDecodeError: Expecting value: line 1 column 1.

Two non-obvious details make the naive fix insufficient:

• Agent bakes its tool schemas into the LiteLLM instance during __init__ (agent.py:677). Setting tools_list_dictionary after construction is a no-op unless the LLM is rebuilt. The patch calls llm_handling(), which is exactly what Agent itself does when it registers planning tools post-construction (agent.py:2058, agent.py:2295).

• The schema alone still fails. With the schema but no director prompt, the model emits a tool call with invented field names and pydantic rejects it:

  1 validation error for HierarchicalOrder
  agent_name  Field required [type=missing,
              input_value={"agent": "Writer", "task": ..., "sequence": 1}]
  

  So director_system_prompt is added too. It goes into short_memory rather than system_prompt, because short_memory_init() (agent.py:1010) has already copied system_prompt into the conversation by that point. Appending to the attribute afterwards would be another silent no-op.

2. Both step() and run() swallow the resulting exception.

step() logged and then fell off the end of its except block, returning None, contradicting its own docstring which documents Raises: Exception: If step execution fails. The inner handler in run() then logged again and continued the loop, and run() returned the conversation. The caller got a successful return containing no work.

Fixing only one of these changes nothing: with just the schema fix a genuine failure is still silent, and with just the raise fix the documented pattern still cannot work.

Changes

• reliability_checks(): attach the SwarmSpec schema to a caller-supplied director, rebuild its LLM, and seed the director prompt into short_memory. All three are guarded with getattr, so duck-typed stand-ins without llm/short_memory (as used in the existing tests) are unaffected.
• step(): re-raise after logging, matching its documented contract.
• run(): count failed loops and raise RuntimeError only when every loop failed. Partial success is preserved, so if some loops succeed their output is still returned.

Tests

Two tests added to tests/structs/test_hierarchical_swarm.py, both offline, reusing the existing StubAgent and make_recovery_swarm helpers:

• test_custom_director_receives_swarmspec_schema — a caller-supplied director ends up with a plan/orders schema.
• test_run_raises_when_every_loop_fails — a run where no loop succeeds raises instead of returning an empty history.

Both were confirmed to fail against unpatched hiearchical_swarm.py and to pass with the patch. The 7 pre-existing offline tests in that file still pass.

Verified end to end against claude-haiku-4-5: case B above now returns roles [Writer, Checker, Director, System] and ~5000 characters.

Note, not fixed here

run_director() sets self.director.tools_list_dictionary = None when planning_enabled=True without rebuilding the LLM, so the tools stay live in the already-built LiteLLM instance. Same class of latent bug, left alone to keep this diff scoped.

🤖 Generated with Claude Code

---

Update — test file dropped. This is a small change to one function, so it ships without a test per the repo's preference for keeping diffs to the fix itself. The verification described above was run directly (reproduction before, same reproduction after); nothing about the fix or the evidence changed, only the absence of a committed test file.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

Hello there, thank you for opening an PR ! 🙏🏻 The team was notified and they will get back to you asap.

[ayaangazali]

Follow-up: this affects three shipped examples, not just the docs. I had originally justified this against the CLAUDE.md snippet, but the repo's own examples use the same pattern:

• examples/multi_agent/hierarchical_swarm/example_hierarchical.py:15,52 — director = Agent(...) → director=director
• examples/multi_agent/hierarchical_swarm/hierarchical_swarm_example.py:99 — director=llm
• examples/multi_agent/hierarchical_swarm/hs_stock_team.py:182 — director=director_llm

All three hardcode gpt-5.4, so I ran example_hierarchical.py unmodified except sed 's/gpt-5.4/claude-haiku-4-5/g'.

On master:

ROLES: ['Director', 'System']

On this branch:

ROLES: ['Research-Team', 'Development-Team', 'Marketing-Team',
        'Director', 'Director-Agent', 'System']

Research-Team, Development-Team and Marketing-Team are the three worker agents the example defines. On master none of them execute.

There is a detail here worth calling out, because it explains how this survived unnoticed and it is arguably worse than the empty return in the original report.

That example uses max_loops=2. Loop 1 fails to parse and contributes nothing. But run_director() adds the director's raw output to the conversation before parsing:

self.conversation.add(role="Director", content=function_call)

So loop 2 returns the director's prose — and the director, being a capable model asked to plan a project, produces a long, structured, confident-looking deliverable. Roughly 8KB of phase breakdowns, sprint tables, risk matrices and success metrics.

It reads exactly like a successful hierarchical swarm run. It is one agent talking to itself, with all three specialists idle. A user eyeballing that output has no reason to suspect anything is wrong, which is likely why this was never reported: the failure mode is not an empty result, it is a plausible result produced by the wrong number of agents.

Same three-line reproduction as in the description, if it is easier to check directly:

sed 's/gpt-5.4/claude-haiku-4-5/g' \
  examples/multi_agent/hierarchical_swarm/example_hierarchical.py > /tmp/ex.py
python /tmp/ex.py | grep -o "'role': '[^']*'" | sort -u

[github-advanced-security]

Pyre found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

[ayaangazali]

On the Pyre summary ("more than 20 potential problems"): those are file-scoped, not diff-scoped. Pyre reports every alert in a file this PR touches, and swarms/structs/hiearchical_swarm.py is 1000+ lines with a long-standing set of them — the Pyre job is red on unmodified master too (3ff34758), alongside Python package, Run Tests and Test Main Features. I cannot read the individual alerts to confirm which are attributable (the code-scanning API needs repo-admin scope), so instead I verified the behaviour this PR claims, directly:

d = Agent(agent_name="MyDirector", model_name="gpt-4o", max_loops=1)
print(d.tools_list_dictionary)          # [] — no SwarmSpec schema
s = HierarchicalSwarm(name="t", agents=[...], director=d)

before: tools_list_dictionary = []
after : schema attached = True | name = SwarmSpec
director prompt injected = True

That is the whole bug: a caller-supplied director arrives with no SwarmSpec schema, so it answers in prose, parse_orders finds no plan, and the swarm returns nothing while reporting success. All three steps of the fix are needed — schema, LiteLLM rebuild (the schema is baked in at __init__, so it has to be re-handled), and the director prompt (without it the model invents field names like agent for agent_name and HierarchicalOrder rejects the order).

Also on this branch now: the test file is dropped and it is rebased on today's master, so the diff is 38 lines in one file.

Worth noting the red checks here are not this PR's — see #1812, which fixes the two workflow defects that make them red for everything (pytest collecting examples/ without the package installed, and a hardcoded laptop path in test-main-features.yml).

[ayaangazali]

Rebased onto master (9d8f6ef) and trimmed the comments to the load-bearing lines. While re-verifying I found something the PR should state plainly rather than leave for CI to surface.

Six existing tests in tests/structs/test_hierarchical_swarm.py fail with this change, and they pass on master only because of the bug it fixes.

test_hierarchical_swarm_execution
test_hierarchical_swarm_multiple_loops
test_hierarchical_swarm_collaboration_prompts
test_hierarchical_swarm_with_dashboard
test_hierarchical_swarm_real_world_scenario
test_hierarchical_swarm_autosave_saves_conversation_after_run

They construct real Agents on gpt-5.4 with no mocking, so without provider credentials the director call returns an error string, parse_orders raises JSONDecodeError, and every loop fails. On master the run then returns the conversation, and the assertion is:

result = swarm.run("Develop a comprehensive strategy …")
assert result is not None

which passes on a run that produced no agent output at all — the exact behaviour #1740 reports. With this change the same situation raises:

RuntimeError: Hierarchical swarm produced no output: all 1 loop(s) failed.

So the six failures are the fix working, on tests whose only assertion was satisfied by the silent failure. Master's own baseline for this file is 8 failed, 17 passed (the arun_stream group, unrelated); with this branch it is 14 failed, 11 passed, and the delta is exactly those six.

Two ways to land it, your call:

1. As-is. These tests need live credentials to mean anything; under the fix they fail loudly instead of passing vacuously. Reasonable if the suite is expected to be run with keys.
2. Tighten the six tests as part of this PR — assert on real output when a key is present and skip otherwise, the way the pure-unit tests in this repo do.

I did not change them here, because rewriting six tests to keep a suite green is the kind of edit that should be a deliberate decision rather than a quiet part of a bug fix. Say which you prefer and I will push it.

[ayaangazali]

Rebased onto 07f3bd39 (was 15 behind). black==24.2.0 --check . and ruff==0.2.1 check . clean.

Flagging something before you merge this, because it will look like a regression and I do not think it is one.

Six tests in tests/structs/test_hierarchical_swarm.py pass on master and fail on this branch:

test_hierarchical_swarm_execution
test_hierarchical_swarm_multiple_loops
test_hierarchical_swarm_collaboration_prompts
test_hierarchical_swarm_real_world_scenario
test_hierarchical_swarm_with_dashboard
test_hierarchical_swarm_autosave_saves_conversation_after_run

Not a stale base — I rebased first and re-ran; the delta is this diff.

They fail because of the RuntimeError this PR adds when every loop fails. With no OPENAI_API_KEY present the director returns a provider error string, parse_orders hits json.loads on non-JSON, every loop raises, and the run now says so instead of returning quietly.

The part worth your attention is what that means about the tests. Here is test_hierarchical_swarm_execution in full, after four Agent constructions and a swarm.run(...):

    # Verify result structure
    assert result is not None
    # HierarchicalSwarm returns a SwarmSpec or conversation history, just ensure it's not None

That is the whole assertion. On master, with no credentials, every loop fails and the test still passes — because the silent-failure path this PR removes hands back a non-None conversation. The test cannot distinguish a working swarm from one that did nothing at all, which is the exact defect this PR is about, showing up in the suite rather than in a user's output.

So the six failures are the fix doing its job on tests that were only green by accident. I have deliberately not touched them:

• weakening the RuntimeError to keep them green would delete the fix;
• rewriting six network-dependent tests belongs in its own change, not smuggled into a one-file bugfix.

Your call on what happens to them. If you want, I will send a follow-up that gives them a stubbed director so they assert something real and run offline — that is a test-quality change and reviewable on its own terms.

Worth noting the whole test job is red on master today (build (3.10/3.11/3.12), pyre, test, test-main-features all fail on 07f3bd39 itself), so this delta is not visible in CI either way — I only found it by diffing the failure set locally against a clean master worktree.

[ayaangazali]

Follow-up to the above — this PR and #1886 add the same mechanism twice, so they should not both merge as-is.

Both add the identical raise to step()'s except, and both add an all-loops-failed guard to run(): this one via failed_loops + RuntimeError, #1886 via any_loop_succeeded + raise last_error. They will conflict textually, and doubling the failure path if both landed is the same hazard as the #398/#402 pair on Swarms-API.

What is unique to each:

• here: the caller-supplied director gets the SwarmSpec schema and the director system prompt — which is the bug this PR is titled for
• [Bug] HierarchicalSwarm reports a total director outage as a successful run #1886: the per-loop conversation marker stops saying "completed" for a loop that raised, and stale last_output is dropped rather than fed forward

Suggested order, your call: merge #1886 first as the general "stop swallowing the error" change, then I rebase this one onto it and drop the now-redundant failed_loops/RuntimeError, leaving it as the focused director-schema fix. Same note left on #1886.

The six failing tests I described above are common to both PRs and have the same cause.

[ayaangazali]

Six tests in tests/structs/test_hierarchical_swarm.py fail with this PR applied, and I want to be clear that they are failing correctly — but they need a decision before this can merge.

master:     8 failed, 21 passed
this PR:   14 failed, 15 passed

newly failing:
  test_hierarchical_swarm_execution
  test_hierarchical_swarm_multiple_loops
  test_hierarchical_swarm_collaboration_prompts
  test_hierarchical_swarm_real_world_scenario
  test_hierarchical_swarm_with_dashboard
  test_hierarchical_swarm_autosave_saves_conversation_after_run

All six build real Agents with model_name="gpt-5.4" and call swarm.run(...). With no API key the director call fails, parse_orders gets a non-JSON body, and:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

On master that exception is swallowed, run() returns the conversation transcript, and the test's only assertion —

assert result is not None

— holds. It passes in 1.25s, which is not four LLM round-trips. So the test currently passes against a run that produced nothing at all.

That is this PR's bug, demonstrated by the suite itself: a total director outage reported as a successful run. Re-raising makes the six stop lying, which is why they now fail.

I have not touched them, because assert result is not None cannot tell success from total outage either way, and there are two legitimate fixes: give them a stubbed director so they run offline and assert something real, or mark them as requiring an API key so they skip rather than pass vacuously. That is a call about the suite, not about this fix, and it affects a file this PR does not otherwise own.

Say which you prefer and I will do it here. Same six fail identically on #1741, which changes the same failure path.

For the record, the change itself is clean:

black 24.2.0 / ruff 0.2.1   clean
import swarms               ok

[ayaangazali]

Brought up to date by merging master in (no rebase, no force-push), and narrowed to one fix.

What changed in scope. This PR previously did two things: backfilled the SwarmSpec schema on a caller-supplied director, and added failure tracking to step() / run() so an all-loops-failed run raises instead of returning a transcript. That second half duplicates #1886, and the two were going to conflict on exactly those lines — whichever landed second would have had to unpick the other. So I've dropped it here and left it to #1886, which does it better: it re-raises the original exception, preserving its type and traceback, where this PR wrapped everything in a generic RuntimeError that threw away both. I also removed the explanatory comments the original diff added.

What's left is the fix this PR is actually about, and it's now a single hunk.

The bug. setup_director() builds a director with tools_list_dictionary=[SwarmSpec schema], base_model=SwarmSpec and the director system prompt. Pass your own director= and none of that is applied — reliability_checks only fills in a director when one is None. The model then answers in prose, parse_orders finds no structured plan, and the swarm returns having done no agent work.

Confirmed on current master:

MASTER: director tools_list_dictionary = []

The fix backfills only what's missing, and only when the caller didn't supply it: the schema, a rebuilt llm (Agent bakes tool schemas into LiteLLM at __init__, so the schema only reaches the model once that instance is rebuilt), and the director prompt into short_memory rather than system_prompt, since Agent has already copied system_prompt into its conversation by then.

Verification on this branch:

before: tools_list_dictionary = []
after:  backfilled = True
schema name: SwarmSpec
prompt injected: True

tests/structs/test_hierarchical_swarm.py — 8 failed / 20 passed, an identical failure set to master, so no regression. Those 8 are pre-existing.

#1886 and this PR now touch different parts of the file and can land in either order.