perf: defer litellm and mcp out of import swarms (2.2s -> 0.37s) - #2090
Open
kyegomez wants to merge 1 commit into
Open
perf: defer litellm and mcp out of import swarms (2.2s -> 0.37s)#2090kyegomez wants to merge 1 commit into
import swarms (2.2s -> 0.37s)#2090kyegomez wants to merge 1 commit into
Conversation
`import swarms` took about 2.0 s and loaded 3,564 modules. litellm accounted
for roughly 1.4 s of that and mcp for another 0.4 s, and neither is needed to
import the package: litellm is needed at the first LLM call, mcp only when a
caller touches MCP.
Both are now deferred to first use. Median over seven interleaved runs:
import swarms 1973 ms -> 388 ms (5.1x, 1.6 s saved)
modules loaded 3564 -> 1408
litellm loaded True -> False
mcp loaded True -> False
The single largest cost was not an import statement. agent.py annotated
reasoning_effort: Literal[get_reasoning_efforts()] = None
and that call runs when the class body executes, importing litellm to widen
the accepted values. Measured against the installed litellm it added nothing
over the static REASONING_EFFORTS tuple, and a type checker cannot evaluate a
runtime call either, so the dynamic form cost 1.4 s and bought nothing. The
annotation now uses the static tuple.
The rest is function-local imports at the call sites in litellm_tokenizer,
litellm_wrapper, agent, llm_manager, context_compressor, agent_loader,
tree_swarm and agent_router, keeping the documented top-level litellm names
rather than internal submodule paths.
mcp is deferred with PEP 562: swarms/tools/__init__ resolves MCPManager and
MCPFileTokenStorage on attribute access, and they leave __all__ so the
`from swarms.tools import *` in swarms/__init__ does not resolve them straight
back. A matching __getattr__ on swarms/__init__ keeps `swarms.MCPManager`
working, and __dir__ still advertises both.
Six tests patched these symbols on the importing module rather than on
litellm, which no longer works now the imports are function-local. They now
patch litellm.completion and litellm.utils.supports_function_calling, which is
where the names actually live and what the tests meant to stub.
Callers who patch swarms.structs.agent.supports_function_calling or
swarms.agents.context_compressor.completion need the same change.
Agent construction still imports mcp, because Agent.__init__ builds an
MCPManager unconditionally. Making that lazy needs mcp_manager to become a
property and is left for a follow-up.
Fixes #1754
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return self.usage_ratio(agent) >= self.threshold | ||
|
|
||
| def _summarize(self, agent: Any, history: str) -> str: | ||
| from litellm import completion |
Comment on lines
+316
to
+320
| from litellm.utils import ( | ||
| supports_function_calling, | ||
| supports_parallel_function_calling, | ||
| supports_vision, | ||
| ) |
Comment on lines
+516
to
+520
| from litellm.exceptions import ( | ||
| AuthenticationError, | ||
| BadRequestError, | ||
| InternalServerError, | ||
| ) |
| dynamic_context_window: bool = True, | ||
| show_tool_execution_output: bool = True, | ||
| reasoning_effort: Literal[get_reasoning_efforts()] = None, | ||
| reasoning_effort: Literal[REASONING_EFFORTS] = None, |
Comment on lines
+1336
to
+1340
| from litellm.exceptions import ( | ||
| AuthenticationError, | ||
| BadRequestError, | ||
| InternalServerError, | ||
| ) |
| config: Union[BaseModel, Dict[str, Any]], | ||
| ) -> AgentConfigDict: | ||
| """Validate a config supplied as a pydantic model or a plain dict.""" | ||
| from litellm import model_list |
| Returns: | ||
| List[float]: The embedding vector as a list of floats. | ||
| """ | ||
| from litellm import embedding |
| Returns: | ||
| List[float]: Embedding vector | ||
| """ | ||
| from litellm import embedding |
| # Set fallback encoder | ||
| fallback_model = default_encoder or DEFAULT_MODEL | ||
|
|
||
| from litellm import encode |
| @lru_cache(maxsize=None) | ||
| def _model_supports_vision(model: str) -> bool: | ||
| """Cached litellm.supports_vision lookup (pure function of model name).""" | ||
| from litellm import supports_vision |
kyegomez
changed the title
import swarms is 5x fasterimport swarms (2.2s -> 0.37s)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1754
import swarmstook ~2.2 s and loaded 3,564 modules.litellmwas ~1.4 s of that andmcpanother ~0.4 s, and neither is needed to import the package.What this actually changes — read this before the numbers
The cost moves; it does not disappear. Measured end to end, from process start to first completion, with a stubbed LLM so no network is involved:
import swarmsAgent(...)constructrun()So for a script that imports swarms and immediately builds an agent, this PR buys nothing. The import cost simply relocates into
Agent.__init__, which builds anMCPManagerunconditionally (pullingmcp) and runsreliability_check(), which callssupports_function_callingandget_max_tokens(pullinglitellm) — both on every agent, whether or not MCP or the model is ever used.The original title claimed "5x faster" without that qualification. It is 5x on
import swarmsalone, which is the honest framing.Where it does pay off
Any process that imports swarms and does not immediately construct an agent:
swarms --helpand other CLI paths that print and exit — ~1.8 s, the most visible win.Conversation, or a non-LLM structure.Import-time result
Median over seven interleaved runs (alternating master/branch, so drift cannot favour either side):
import swarmslitellmloadedmcploadedMin-to-min is 1668 → 376 ms, so it is not a sampling artifact.
The largest cost was not an import statement
agent.pyannotated:That call runs when the class body executes, and
get_reasoning_efforts()imports litellm to widen the accepted values. Two things make it not worth 1.4 s:REASONING_EFFORTStuple (litellm adds: nothing).The annotation now uses the static tuple. That single change took
import swarmsfrom 2564 ms to 1148 ms and made litellm disappear. Each preceding round of function-local imports had barely moved the needle — this was the anchor holding the chain.The rest
litellm — function-local imports in
litellm_tokenizer,litellm_wrapper,agent,llm_manager,context_compressor,agent_loader,tree_swarm,agent_router. These keep the documented top-level names (from litellm import completion) rather than internal paths likelitellm.main: the submodule form measures identically, since Python initialises the parent package regardless, and it couples us to internals that move between versions — the waymcp.server.fastmcpvanished in mcp 2.x.mcp — PEP 562.
swarms/tools/__init__resolvesMCPManagerandMCPFileTokenStorageon attribute access, and they leave__all__so thefrom swarms.tools import *inswarms/__init__does not resolve them straight back — that star-import defeated the first attempt. A matching__getattr__onswarms/__init__keepsswarms.MCPManagerworking, and__dir__still advertises both.Every previously working access path was verified:
swarms.MCPManager,from swarms.tools import MCPManager, the direct submodule import,dir(),count_tokens,LiteLLM, andAgent(...).Breaking change
Six tests patched these symbols on the importing module, which stops working once the import is function-local:
User code doing the same needs the same change. This belongs in the release notes.
Verification
No regressions across the eight affected suites, from a detached worktree, run offline:
Identical failure sets —
commreports nothing introduced. The 33 are pre-existing live-LLM tests needing an API key.black --line-length 70andruff check .clean.Required follow-up for this to reach users
This PR is the groundwork, not the payoff. To move the ~1.8 s off
Agent()construction as well:Agent.mcp_managera lazy property so agents without MCP never import it.reliability_check()probes (supports_function_calling,get_max_tokens) to first use.Then construction stays cheap and the cost lands on the first genuine model call, overlapped with network latency. Merging this first is what makes that follow-up possible.