feat(agents): hosted agent runtimes — stateless execution, Entra ID auth modes, discovery, and tools as workflow tasks - #1582
Open
v1r3n wants to merge 49 commits into
Open
feat(agents): hosted agent runtimes — stateless execution, Entra ID auth modes, discovery, and tools as workflow tasks#1582v1r3n wants to merge 49 commits into
v1r3n wants to merge 49 commits into
Conversation
…eployments
AzureFoundryAgentClient stored per-run state (endpoint, runId, token provider)
in an in-memory ConcurrentHashMap keyed by threadId. In multi-replica server
deployments the 5-second status-poll callback could arrive on a different pod
than the one that ran startAgent, causing "No execution found" failures on every
execution after the first.
Fix: startAgent now returns a compound executionId that base64-encodes the
non-sensitive run context {threadId, runId, endpoint, assistantId, apiVersion}.
getAgentStatus decodes this to reconstruct the Azure API call on any pod and
re-authenticates using the original task credentialRef, which is available in
the ConductorAgentRequest passed by the updated ConductorAgentDelegate.
Interface change: ConductorAgentClient.getAgentStatus(String) gains a second
ConductorAgentRequest parameter. All five implementations and both test fakes
are updated. ServiceConductorAgentClient ignores the new parameter (AgentService
has its own persistent state). BedrockAgentClient signature is updated; its
in-memory model is unchanged pending a follow-up — the stateless fix for Bedrock
requires a similar compound-executionId approach.
The respondContexts map is retained for respond() and cancelAgent(), which
receive no credentialRef and therefore cannot re-authenticate on a different
replica; single-turn agents (the common case) no longer use it.
…RunStore SPI Following the same pattern as ServiceConductorAgentClient -> AgentService, run context (threadId, runId, endpoint, credentialRef, scope) is now persisted in an injected AzureAgentRunStore rather than encoded in the executionId. The default store is in-memory (InMemoryAzureAgentRunStore, @ConditionalOnMissingBean); HA deployments can substitute a Redis or DB-backed implementation, mirroring how orkes-conductor overrides SkillMetadataDAO and SkillPackageStore. This also removes the previous same-pod limitation on respond() and cancelAgent(): by storing credentialRef in the run context, any replica can re-authenticate and handle multi-turn and cancellation requests without in-process state.
Brings the branch up to date with main (96 commits) and resolves the overlap: - AzureFoundryAgentClient: kept the stateless rewrite, and ported main's 52ab3da fix into AssistantsRunApi.extractText, where that logic now lives. Assistants with code interpreter return an image_file part ahead of the text part, so content[0].text was empty for them. Covered by a new test. - Test fixtures now type their content parts, as the real Assistants API does. - a2a-integration.md: kept main's wording, corrected the runtime list. - conductor-agents.md, mkdocs.yml, .gitignore: took main's versions; the docs nav entry was re-added under main's restructured Agents > Build section.
…ssumeRole, OBO Combines PR #1447 into this branch. Where the two overlapped, #1447's auth and discovery work is ported onto the stateless client structure here rather than the stateful one it was written against. Azure auth (new AzureFoundryAuth): - Four credential modes, first match wins: api-key header, service principal, user-assigned managed identity, default credential chain. A deployment on managed identity now needs no credentialRef at all. - Caller identity (OBO): exchanges the caller's Entra token for a Foundry-scoped one. Never cached — it belongs to a person, not the deployment — and falls back to credential auth when the service principal to exchange it is absent. - Scope follows the endpoint (ai.azure.com / ml.azure.com / cognitiveservices), overridable, and resolved only on a cache miss so a poll reads no secrets. - AssistantsRunApi now takes an AssistantsAuth rather than a bearer string, since Azure may authenticate by api-key header. OpenAI supplies a bearer key. Discovery: agents visible to a credential appear in the agent list — a secret with an 'endpoint' key lists Azure agents, one with 'region' lists Bedrock. Both clients gained listExternalAgents/getExternalAgentDef; AgentService scans secrets for them. Bedrock: AssumeRole (roleArn, roleSessionName, externalId) alongside static keys and the default chain, and the SDK client cache key no longer resolves secrets to compute. agentUrl as a top-level field for both: Azure splits a trailing /assistants/asst_x or /agents/NAME off as the agent; Bedrock parses bedrock://AGENTID/ALIASID?region=. Kept from this branch where they conflicted: the stateless execution model, the parallel-tool contract, and autoRunTools. Kept from main: the agent_capabilities to tags mapping, which #1447 dropped incidentally. Tests: the Azure suite moved to api-key auth so it runs offline — the Identity SDK uses its own HTTP stack, so an OkHttp interceptor no longer catches token calls. New AzureFoundryAuthTest covers mode selection and the OBO exchange against a mock Entra endpoint. Their ITs updated for the two-arg getAgentStatus. UI: took #1447's two genuine fixes — dropped 'unresolved' from the task card, since a workflow registered outside the editor never has a snapshot, and made the live input's agentType authoritative over a lagging snapshot. Provider logos, filter chips, and the caller-identity toggle come in on top of the five-runtime typing already here.
Completes the #1447 fold-in. Foundry serves three APIs behind one agentType and they do not share a protocol; only the classic Assistants surface was handled. - Model inference (chat completions) and a project's Responses API both answer inside the start call, so they report a terminal state through ConductorAgentStartResponse rather than being polled — the same mechanism Bedrock uses, which makes them stateless without a run to remember. - A project agent's own instructions and tools are read from its definition and forwarded, so web search, code interpreter and file search actually run. code_interpreter is wrapped in the container object the Responses API requires. - getAgentStatus reports terminal for both, and respond rejects them with a clear message rather than quietly issuing thread operations against an endpoint that has no threads. rawConfig.surface (assistants | responses | inference) overrides the hostname inference. #1447 classified purely by hostname, which silently misroutes sovereign clouds (.azure.us, .azure.cn), private endpoints and proxies to the classic path — and made the behaviour untestable against a local server.
v1r3n
changed the title
The suite runs on api-key auth, so nothing reaches an Entra token endpoint through OkHttp any more — the Identity SDK uses its own HTTP stack. The interceptor was still installed but no longer asserted on, and its comment described an exchange the client no longer performs.
…et name
Conductor already substitutes ${workflow.secrets.X} in task input before the task
runs — ParametersUtils.substituteSecrets, applied at AsyncSystemTaskExecutor:194
and restored afterwards so nothing is persisted. AnnotatedWorkflowSystemTask is
async, so the AGENT task goes through exactly that path.
The hosted-agent clients ignored it: they took credentialRef as a secret *name*
and called CredentialResolutionService themselves, reaching into the secret store
from inside task execution.
That made the platform's own syntax dangerous. "credentialRef":
"${workflow.secrets.AZURE_CRED}" substitutes to the whole secret, so the client
then looked up a secret named by the JSON blob, found nothing, and every sub-key
resolved null — dropping Azure through its whole chain to DefaultAzureCredential.
No error: the agent silently ran as the host's own identity. The only spelling
that worked was a bare name, the one that bypasses the platform.
- credentials (Map<String,String>) replaces credentialRef on the start, respond
and cancel requests. A workflow names each field and the engine substitutes it,
matching how an HTTP task takes an Authorization header.
- All four clients drop CredentialResolutionService. A test asserts none of them
can hold or be given one, so this does not quietly come back.
- Reading a credential rejects any value still holding ${workflow.secrets.…}.
Conductor does not substitute for input held in external payload storage
(AsyncSystemTaskExecutor:189-193 warns about this), and running as the host
identity is worse than failing.
- Discovery is a control-plane scan with no task behind it, so AgentService keeps
reading secrets — but it already parsed them, so it now hands clients values too.
- The AZURE_FOUNDRY_ENDPOINT fallback is gone; an endpoint kept in a secret is
written as ${workflow.secrets.NAME} like anything else.
- Azure's managed-identity key is renamed clientId -> managedIdentityClientId.
It sat beside client_id meaning something different, so a plausible typo skipped
the service principal and authenticated as the wrong identity.
Caching now counts auth resolutions rather than secret reads: with values handed
in there are no reads to count, and what the cache actually prevents is rebuilding
the SDK credential — which would discard the token it caches internally.
…e one Follows the credentials rework by answering 'does this provider take an API key' consistently. - Azure wrote apiKey and OpenAI wrote api_key for the same thing, so configuring a second provider meant discovering that. Both spellings now work wherever an API key is accepted. - Bedrock silently ignored an apiKey and fell through to the host's AWS credential chain — the same silent wrong-identity failure the credentials rework removed elsewhere. It now fails with the reason: Bedrock API keys are bearer tokens and the bundled AWS SDK (2.31.68) signs Bedrock Agent Runtime calls with SigV4, its client builder exposing no token provider. - Reading credentials moves out of AzureFoundryAuth into a shared AgentCredentials, since Bedrock and OpenAI were both reaching into an Azure-named class for it. The unresolved-reference guard now lives in exactly one place. - The docs gain an at-a-glance auth matrix per provider. Unchanged deliberately: AgentspanAIModelProvider still resolves LLM provider keys from the secret store. It looks them up by a fixed provider name (OPENAI_API_KEY and friends), not by anything a workflow supplies, and serves the provider-status endpoint as well as task execution — so no user-controlled secret name flows through task input and the double-resolution bug cannot arise there.
I said this was blocked on the AWS SDK. It was not — I checked one door (no tokenProvider on the client builder) and stopped, having already seen authSchemeProvider on the same builder. A Bedrock API key is a bearer token, and this service's model declares only aws.auth#sigv4, so the SDK signs the request and ignores the key. Supplying a custom auth scheme provider that resolves to smithy.api#noAuth stops the signer running; anonymous credentials stop the default chain being probed; and an execution interceptor carries Authorization: Bearer. All three pieces are in the bundled SDK (2.31.68) — no dependency bump, no change for the five other modules sharing revAwsSdk. An API key now takes precedence over static keys, AssumeRole and the default chain, and either spelling selects it. Tested against a real request rather than by inspecting the builder: the client is pointed at a local server and the sent request is asserted to carry the bearer token and no AWS4-HMAC-SHA256 signature. Only the request on the wire proves the signer actually stood down.
The AGENT form still asked for a "credential reference" - a bare secret name
the server read itself. After the credentials rework nothing reads that field,
so the input was silently doing nothing: a task configured through the form
could not authenticate.
Replace it with the auth methods each runtime actually supports. Picking one
shows only its fields, and a stored-secret picker fills them with
${workflow.secrets.NAME.key} references, so the author never types the
substitution syntax or guesses which keys a provider wants. Switching methods
clears the other methods' keys, so a half-filled service principal cannot
shadow an API key.
The method catalogue mirrors the server's resolution order, so what the form
offers is what the client will pick.
Snapshots now carry the auth method rather than a credential name - a snapshot
describes the agent, and the values are secret references resolved at run time.
…reen The screen used one word for the provider's own credential (Azure's client_secret, an API key) and for the Conductor store entry that holds it, so "The secret holds client_id, client_secret and tenant_id" read as nonsense. Name the store entry for what it is, and say what each Azure field means in the terms the Azure portal uses. Also drops an effect that reset the chosen method on runtime change: the choice now carries the runtime it was made under and is discarded during render, so switching provider cannot paint the previous one's fields.
…hing "Token scope (optional)" appeared with no explanation, and appeared even under API key auth - which sends a header and mints no token, so the field could never be read. Hide it unless the chosen method actually mints a token, and say what a scope is: which Azure resource the token is valid for, derived from the endpoint host unless the deployment hides it.
…esolve
A ${workflow.secrets.NAME.key} reference resolves to null when the secret is
missing or is not JSON holding that key - silently, since the engine only
warns. Every credential then arrives blank, no auth mode matches, and both
Azure and Bedrock fell through to the identity the server itself runs as. The
agent ran as somebody else and the only symptom was a confusing failure from
the default credential chain, or worse, a call that succeeded with the wrong
privileges.
The existing guard did not cover it: it fires on a value that still holds a
literal ${workflow.secrets....}, and here the values are null.
Fall back only when the task supplied no credentials at all, which is how
managed identity and instance roles are meant to be configured. When it named
credential keys and none of them resolved, say so and name them.
This makes a half-configured service principal an error where it previously
degraded to the deployment identity - the same silent downgrade, since it also
ignores a request to act as the caller.
${workflow.secrets.NAME.key} extracts a key from a JSON secret, and a correct
JSON document reaches us unreadable in two routine ways: a .env file read
verbatim keeps the quotes a shell would have stripped, so the value begins '{
rather than {; or the document was JSON-encoded on the way in and is a JSON
string holding JSON. Either way the parse failed and the reference resolved to
null - which, for a credential, means every key arrives blank.
Unwrap and retry, but only after a straight parse has already failed, so a
value that reads correctly is never second-guessed. Warn when it happens: the
stored value is still wrong and should be fixed at the source.
Reviewing the previous two commits turned up four problems.
A flat ${workflow.secrets.NAME} has no JSON to unwrap, so the fix for the
quoted secret never reached it. An API key, and every Bedrock and OpenAI
credential that is stored on its own, still arrived with the quotes attached
and failed at the provider with nothing pointing back at the secret. Reject it
where credentials are read, naming the field. Rejected rather than trimmed: no
credential is quoted on purpose, and sending a guess produces the same opaque
failure.
The unusable-credential error said "none of them resolved" even when some had.
For a partly resolved credential that points at the secret when the problem is
the task - the same misdirection that made the original bug expensive. Name
which keys arrived and which did not.
The retry warning blamed quoting for both cases it rescues, misdiagnosing a
JSON-encoded secret. Say which one it found.
AzureFoundryAuth.resolve still documented the fallback it no longer does, as
did its warn and the Bedrock javadoc. The Vertex placeholder now says to call
the guard before Application Default Credentials, which resolve on almost any
GCP host and would otherwise run the agent as the node's service account.
Also unbalanced braces inside a {@code} tag in the javadoc added last commit.
A second review pass. The credentials rework removed the secret-store read from the auth path, but four comments still justify the caching design by it: resolveScope's javadoc, ProviderKey's, the get-then-put note, and a test's. The design those comments defend outlived the cost - scope resolution was deferred, and the cache keyed on the raw override plus the endpoint, purely so a lookup would not touch the store. Key on credentials and the resolved scope instead. Two endpoints that resolve to one scope now share a token provider, which is correct: a token is scoped to an Azure resource, not a URL. An explicit scope override still gets its own. Also: cancelAgent resolved credentials outside its own try, so a broken credential propagated instead of warning like every other cancel failure (both callers catch, so this changed which warning appeared, not whether cleanup ran); and the unsubstituted-reference check now runs before the quoting guard, being the more specific diagnosis of the two. The ParametersUtils test now uses the spacing a hand-written credential blob actually has rather than a serializer's compact form.
Microsoft renamed the product. Follow it through the UI, the docs, and the agentType wire value, which is public API. agentType is written into workflow definitions people have already saved, so the old string keeps routing rather than breaking them: ConductorAgentClient gains agentTypeAliases(), A2AWorkers registers a client under its aliases as well as its type, and the Foundry client claims microsoft-foundry with azure-foundry as its alias. agentType() is the name reported back and shown, so discovery and the UI both say the new one. The UI normalizes an old stored value through canonicalAgentType, so nothing downstream carries two spellings. Azure, Entra ID, DefaultAzureCredential and the ai.azure.com scopes are untouched: the cloud and the identity platform did not get renamed, only the product. Internal class names (AzureFoundryAgentClient, AzureFoundryAuth) are left alone, being neither public API nor user-visible.
Three claims in the page described behaviour this branch replaced. The failure table still said an absent or incomplete credential "falls through to the next auth mode, ending at the platform's default credential chain", which is the silent wrong-identity path that was removed. It now separates the case that legitimately falls back - no credentials at all - from the ones that now fail, and names the quote-wrapped credential too. The caller-identity section promised that a partial service principal still runs as the service identity. It fails now, and the page says why. "The secret has to be JSON" said a quote-wrapped secret simply breaks. It is recovered from, with a warning, along with a JSON-encoded one; what cannot be recovered is a flat credential, which has no JSON to unwrap. Also: the ten-minute auth cache is keyed by credential and scope, not by endpoint, so the page no longer claims it saves a secret-store read that no longer happens; agentUrl was only documented per-provider though all three take it; a "as with Azure" the rename missed; and a short section on the AGENT task form, which is how most people will configure this. AGENTS.md now records that the docs build needs PYTHONPATH=. - without it mkdocs dies at config parse with "cannot find module 'main'", which reads like a broken checkout.
…is saved Every runtime except A2A reads the message from prompt and nowhere else - ConductorAgentDelegate throws "AGENT requires 'prompt'" - while A2A takes any of message, parts, text or prompt. Until now that was only discovered at run time, on a workflow that may already have done work and, for a hosted agent, after a conversation had been opened on the platform. WorkflowTaskTypeConstraint now checks it at save, alongside the existing per-type rules, so the definition is rejected with the task named. A prompt supplied by the task def's inputTemplate counts, as it does for HTTP. An agentType that is itself an expression is not judged: what it resolves to is unknowable until the workflow runs. The task form marks the field required and says why, so the author sees it before the server does. Its ConductorInput mock no longer swallows helperText and required, which would have hidden the behaviour under test.
An agent's built-in tools - web search, code interpreter, file search - run inside the platform and never set requires_action, so they never appear in pendingTools, which only reports the function calls handed back to the workflow. Nothing else recorded them either, so a run that searched the web and executed code left behind only its final sentence. Both surfaces now report them as executedTools on completion. The Responses API returns one output item per step and only some are messages; extractResponseText kept the message text and dropped every tool item on the floor. The classic Assistants surface does not carry them on the run at all - only the run's steps do - so those are fetched once when the run reaches a terminal state, best effort, since a run that finished correctly should not fail over missing step detail. Each call keeps its own fields rather than being mapped onto a fixed schema: the shape differs per tool and Azure adds new ones. The execution view lists them with the input each was given. Docs gain an Observability section covering both kinds of tool call, and stating plainly that Conductor emits no OpenTelemetry - so a run driven from an AGENT task does not appear in Foundry's own Tracing view, which reads GenAI spans from a connected Application Insights.
On the Responses surface the client read the agent's definition with GET
/agents/{id} and replayed its model, instructions and tools as an anonymous
response. The request named the agent nowhere. Answers came back looking
correct, which is why this went unnoticed, but Azure was never asked to run the
agent - so there is no record of it being called, the run appears in no agent's
history, and none of the project's per-agent monitoring sees it. Anything in
the definition the replay did not copy - model parameters, response format,
attached knowledge, the version - was silently dropped too.
Name the agent instead, per the Foundry REST reference:
POST {project}/openai/v1/responses
{"agent": {"type": "agent_reference", "name": "..."}, "input": [...]}
Foundry then applies the agent's own definition. model and instructions are no
longer sent on this surface, since overriding them detaches the run from the
definition it is attributed to; they still apply to inference. rawConfig gains
agentVersion to pin a version and conversation to join an existing thread.
fetchAgentDefinition and toResponsesApiTools go with it - both existed only to
replay a definition the platform now applies itself.
Verified against Microsoft's REST documentation, not a live project: the shape
is asserted from the recorded request body in AzureFoundrySurfacesTest.
The service rejects "agent" outright: HTTP 400 invalid_payload — The 'agent' property is deprecated. Use 'agent_reference' instead. Microsoft's REST reference still shows "agent"; its Python and TypeScript samples pass agent_reference in the request body, and the live endpoint agrees with those. Same object either way - type and name, plus version when pinned.
An answer that cites the web while reporting no tool call is either a response that did not carry its tool items or an extraction that did not recognise them, and the task output cannot tell the two apart. Log the types at DEBUG - types only, never content, since the reply is the user's own data.
…ncluded startAgent logged the whole ConductorAgentStartRequest at INFO, serialized to JSON. That predates credentials arriving as values: since 1ba7e36 the request carries resolved client secrets and API keys, so every agent call wrote them to the log in clear text. Removed, along with the @SneakyThrows that existed only for its serializer call. The item-type diagnostic now also reports how many of those items read as tool calls, which tells an extraction that found nothing apart from a list lost after it was built. Guarded by isDebugEnabled again. Adds a delegate test covering the whole path a synchronous surface takes: a start response carrying executedTools has to reach the task output, since that is the only write such a task gets.
The worker returns A2ACallResult, not the TaskResult it built, so the task
output is whatever that class declares. Four keys the agent path writes had no
field to survive in and were dropped on the way out, silently and after the
agent code had finished - which is why nothing in that code could show it:
executedTools the tools the platform ran itself, the reported symptom
pendingTools every outstanding call, so a multi-tool turn showed only one
subWorkflowId the child execution a Conductor agent runs as
toolDispatchId worst of the four: the delegate reads it back off the task
output to advance a dispatch, so under autoRunTools it was
never found and every poll started the agent's tools again
A test now walks ConductorAgentResults' KEY_ constants and fails if any lacks a
field here. Verified it fails by renaming one - a mapping this quiet needs a
check that speaks up on its own.
…lready done extractExecutedTools read every non-message output item as a tool the platform had run. A function_call is the opposite: the agent naming a function and stopping until an output comes back. So a Responses-surface agent with function tools reported the request as work done and completed the task without ever running it. Split the two. function_call becomes pendingTools and the turn is WAITING; everything else non-message stays executedTools and the turn completes. The surface can now be continued, which it could not before - respond() threw on it outright. respondWithStatus posts the function_call_output items and returns the next turn directly, the way Bedrock already does, chaining with previous_response_id: the turn's own id is already the executionId Conductor persists, so no second handle has to be carried anywhere. A configured rawConfig.conversation still rides along to group turns in the portal. getAgentStatus re-reads the turn rather than assuming it finished, since a Responses turn answers inside the call but can still be waiting on tools. Model inference genuinely holds nothing to poll and keeps the terminal shortcut.
…ext work scheduleNextIteration can only re-run a statically declared loopOver, and the primitives underneath it are package-private, so a running task had no way to schedule work the definition does not declare. An agent running the tools its model just asked for is exactly that case. The implementation is the sequence scheduleNextIteration and finalizeRerun already compose - getTasksToBeScheduled per task, setTaskDomains, dedupAndAddTasks, scheduleTask - minus the loop-specific parts. Tasks absent from the definition are safe there: getTasksToBeScheduled takes a WorkflowTask object and never consults the def, and a non-terminal task holds the workflow open. What they do not get is a successor, since getNextTask reads the definition. That is deliberate: advancing past them belongs to the task that asked for them, being the one that knows when it has what it needs. Tested including the silent part - a repeated reference name is dropped by dedupAndAddTasks, which is why a caller scheduling several rounds has to distinguish them.
Reviewing the two previous commits adversarially, assuming they were wrong. Ranked by what they would have cost. A transient failure re-ran every tool. advanceToolDispatch cleared toolDispatchId before submitting the results, so a dropped connection left a task that looked as if it had never dispatched anything; the next poll re-read the same outstanding calls and dispatched them again, repeating until the 24h deadline. Making getAgentStatus a real read is what turned that from harmless into duplicate side effects. Clear the handle only once the results are in. An agent named only by agentUrl could not be continued at all. respond and cancel rebuild the target from rawConfig alone, and neither request carried agentUrl, so the first continuation threw "endpoint must be provided". No test caught it because none carried one configuration through both halves of a turn. A rejected turn read as success. The reply carries its own status, and a content filter or an exhausted token budget answers 200 with no message - which the output items alone report as a completed agent that said nothing. queued and in_progress were likewise read as finished. A conversational resume threw. continueResponse always shaped its input as tool results, so resuming with a prompt hit "waiting on 0 tool calls". It now sends a message when nothing is outstanding - but still fails loudly when toolResults is present and empty, which is a broken dispatch rather than a new thing to say. conversation and previous_response_id were sent together; they are alternatives and the pair is rejected, so the feature broke for exactly the deployments that configured it. Assistants run steps counted function calls as tools the platform ran - work Conductor did, credited to Azure, and soon to be recorded twice. executedTools were computed on every waiting turn and written on none, since writeCompleted was their only writer. They now accumulate across turns, so the work done on the way to an answer survives. scheduleDynamicTasks queued into terminal workflows - its own test scheduled into a FAILED one and asserted success. It now refuses, returns what it actually scheduled, and the tests assert the queue push rather than only persistence, and run without a registered TaskDef, which is the real case. Docs still said the Responses surface answers in one call, cannot be resumed, and has no tool loop.
Nothing ran a real client through the real delegate. The client's tests use a fake server and never build a task; the delegate's tests use a fake client and never build a request. Both stayed green while the two halves disagreed about how an agent is located and about what a function call even is - which is how those defects survived a full suite. AzureFoundryToolLoopTest closes that seam: Foundry asks for get_revenue, the delegate schedules it and holds the task in progress, the tool's output goes back as a function_call_output keyed by the call it answers, and the agent finishes. The agent is named by agentUrl alone, so the continuation has to rebuild the target from it - the case that failed outright before. Also documents what has to line up for a tool to run at all, none of which fails loudly today: the agent needs a function tool rather than a built-in one, autoRunTools has to be on, and a worker has to serve the tool's task name or the task simply sits scheduled.
…tself Asked whether Azure can be told not to run a tool and hand it back instead. It cannot: the tool's type decides who runs it, and in Foundry's own catalogue function is the only client-side one - web search, code interpreter, file search, AI Search, OpenAPI and the rest all execute inside the service. So the way to have Conductor do the work is to declare the capability as a function tool, not to ask the platform to stand down. That normally belongs on the agent, where agent_reference picks it up and no workflow change is needed. rawConfig.tools covers the case where the agent cannot be edited: the list rides with the request and its continuations, so the agent asks for a function Conductor schedules as a task. Opt-in and inert by default - absent the key nothing is sent and the agent's own tools apply exactly as before, which matters because sending a partial list would quietly narrow what the agent can do. Rejected outright when it is not an array, rather than serialising something the service will refuse. Documented with the caveat it deserves: whether Foundry treats the list as a replacement or an addition is not settled by its REST documentation, so a run that must not touch a built-in tool should have the agent defined that way.
The tool loop is documented from the Conductor side but never said how to get an agent that asks for a tool in the first place - which is the step that decides whether any of it happens. Covers creating the agent with a function tool over REST, including that the Foundry portal cannot add one (it shows them but has no editor), and what the worker actually receives: the function's arguments flattened into the task input, plus _toolCallId, _toolName and _agentExecutionId, taken from SubWorkflowAgentToolDispatcher.toolTask rather than from memory.
The walkthrough used gpt-4.1-mini as if it were the value Azure resolves. It resolves against the project's deployments instead, so the agent is created happily and fails at the first call with DeploymentNotFound - which says nothing about the agent that caused it. Says so, and gives the two ways to find the right value: read it off an agent that already works, or list the deployments on the resource.
Reviewing Microsoft's function-calling reference against the implementation.
Its guidance is to "treat tool arguments and tool outputs as untrusted input",
and we were not.
Tool arguments become a task's input parameters, and the engine resolves
${...} in those against the running workflow. The arguments are written by a
model, from a prompt that may carry text from anywhere, so
${workflow.input.customer_ssn} in an argument was a request the engine
fulfilled - handing workflow data to the tool as if the author had asked for
it. Escaped with the engine's own $${ convention, nested values included, so
the tool receives what the model actually wrote.
Two constraints the docs were silent about:
A run expires 10 minutes after creation, and that is total elapsed time, not
per function. A tool task slower than that submits to a run that no longer
exists, and maxDurationSeconds does not help because Azure's clock is
independent. Documented with the split Microsoft recommends, which happens to
be the shape Conductor is good at: return a handle, do the long work as
separate tasks.
strict with additionalProperties and required constrains the arguments to the
schema - worth having, since the worker is handed them as its input.
Also removes a test asserting that the default Azure credential chain throws.
That holds only where no Azure login exists, so it passed in CI and failed for
anyone who had run az login - everyone working on this integration.
AzureFoundryAuthTest already pins the mode selection without consulting the
environment.
…ploymentNotFound Azure reports it as a fault of the request that failed. With agent_reference the request carries no model at all - Foundry resolves it from the agent's own definition - so the message names no agent, no model, and nothing that leads back to the definition that caused it. The error now says to check versions.latest.definition.model against the project's deployments, which is the one thing that fixes it.
…efault An agent that asks for a tool is asking the workflow to do work. That work is now a task in the agent's own workflow, and the run stays open until a worker has done it. Two changes make that true. Tool execution is the default rather than something to opt into: absent autoRunTools, a tool call was handed back and the AGENT task completed with waiting: true, so a workflow whose author had said nothing about tools finished green having silently dropped the agent's question. Handing the call back is still available, now as autoRunTools: false. And InlineAgentToolDispatcher schedules those tasks into the agent's own workflow through scheduleDynamicTasks, rather than a child workflow where the agent's execution shows none of the work it did. It holds nothing between calls: the workflow, the agent's task and the turn are all in the dispatch id, so a poll on another replica resolves the same batch by reading the workflow back. Turns are numbered from the workflow rather than remembered, because a repeated reference name is dropped without a word. conductor.integrations.ai.agent.tool-execution chooses between them, defaulting to inline; the two beans cannot both be present, since A2AWorkers resolves the dispatcher by type. Naming, argument parsing and the expression escaping move to AgentToolNaming and AgentToolArguments, shared by both, so a result cannot fail to match the call that asked for it because two dispatchers named things differently.
A2AWorkers resolves the dispatcher by type, so two beans is not a preference conflict - it finds none, and the agent quietly stops running its tools. Both implementations were unit tested happily without anything asking whether the container would produce one, which is the question that decides whether any of it runs. Covers the default (inline), the property that selects the child-workflow one, and neither being present without the AI integration, which is what an SDK worker sees.
…om global agent list - Add `endpoint` field to AgentSummary DTO (Java + TypeScript); populated by AzureFoundryAgentClient.listExternalAgents() so the global /agent/list scan carries the Foundry project URL alongside each discovered agent - AgentTaskForm: extend /agent/list fetch to fire for microsoft-foundry runtime; add "Select agent" dropdown above PROVIDER_FIELDS that pre-fills rawConfig.assistantId and rawConfig.endpoint on selection — user no longer has to copy-paste the project URL after picking an agent
…rom listExternalAgents Add five tests to AzureFoundryAgentClientTest: - listExternalAgentsReturnsAgentNamesFromClassicEndpoint - listExternalAgentsCarriesEndpointOnEachSummary (the property under test) - listExternalAgentsSetsTypeMicrosoftFoundry - listExternalAgentsStripsTrailingSlashFromEndpoint - listExternalAgentsReturnsEmptyListOnConnectionError Extend FoundryDispatcher to handle GET /openai/assistants and GET /agents so the discovery path can be exercised without real Foundry credentials. Also apply spotless formatting to AgentSummary.java.
Contributor
|
Updated conductor-oss with following changes -
Additional :
Here is sample demo : |
One turn worked by accident of the state fitting in a single exchange. Tracing an agent that asks for tools again after seeing a result found eight defects sharing a cause: we kept our own record of where the loop was, in the task's output, and drove off it. A2A keeps only the remote's id and asks it every poll; this moves the same way. The worst was invisible. advanceToolDispatch cleared its handle with remove(), but an agent task's output reaches the store by merging the returned object over what is there, and null fields are omitted from that merge - so the key was never cleared. When the provider answered a submit with "still working", the next poll submitted the same tool outputs again, the provider rejected them, and thirty poll failures later the task died claiming the agent was unreachable. Emptied rather than removed now, since an empty value overwrites where a missing one does not. A retried tool kept its reference name, so the workflow held the failed original beside the new attempt and one failure was fatal however many retries remained - and the running retry was then cancelled. Only the latest attempt counts now. A failed tool is reported to the agent as that call's result rather than failing the batch: it can try another tool or say it could not find out. Tool tasks are optional, so an exhausted retry marks the task COMPLETED_WITH_ERRORS instead of terminating the workflow before the agent could be told. Also: a dispatch that could not schedule everything now fails the task rather than being waited on, since a partial batch would answer the provider with tool calls missing; the dispatcher reads the execution store rather than a source that falls back to the search index; cancel dequeues before marking CANCELED so no worker runs an abandoned tool; a missing tool call id is refused rather than keyed on the string "null"; poll failures reset on any successful poll; an output offloaded to external storage fails loudly instead of starting a new provider run every five seconds; the idempotency key includes the retry count; and maxToolTurns (default 10) bounds an agent that keeps asking. Tool tasks are named for the tool - agent_ref__t1__get_revenue - since that is what shows in the task list. Nothing correctness-bearing hangs off the name.
… no worker An agent's tool tasks could not be in the workflow definition - which tools run, and how many rounds of them, is the model's decision - so the diagram had nowhere to draw them and they appeared as siblings of the agent that asked. They are in the execution though, named after that agent, which is enough to nest them: the same trick the dynamic fork plays with forkedTaskDefs, needing no new data from the server. An agent with tools is drawn as a container of them, reusing the loop container rather than inventing a second agent diagram; one that ran none stays an ordinary card, since a container with nothing to nest is an empty box round a leaf. Children are ordered by turn number, not by name, or turn 10 would sort between 1 and 2. Separately, a task that is scheduled with no polls now says so. Nothing is listening for that task name, which is a plain fact about any task and the usual reason an agent looks stuck - it asked for a tool nobody serves. The task staying scheduled is correct; leaving someone to work out why was not. Docs cover the loop as it now behaves: a failed tool is reported to the agent rather than failing the workflow, turns are bounded by maxToolTurns, and tool tasks are named for the tool and the turn.
…viour clearToolBatch writes toolDispatchId="" and pendingTools=[] rather than removing the keys (empty value overwrites; absent key does not, so the stale dispatch id would survive the output merge). The assertion was still checking doesNotContainKey, which fails now that the keys are present with empty values.
With retryCount=3 on the AGENT task (the worst-case on a long-lived CI server whose task-def cache has not been refreshed), each of the four attempts starts a fresh sub-workflow. The LLM task inside it retries three times with exponential backoff (2+4+8=14s delay), so each attempt takes ~16s — 4 × 16s ≈ 64s, which exceeds the previous 60s timeout. Verified locally: test passes against a running server in ~80s.
shaileshpadave
force-pushed
the
feat/hosted-agent-tool-execution
branch
from
September 2, 2026 10:36
b903241 to
27464cc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request type
Changes in this PR
Makes the hosted-platform agent runtimes correct across replicas, implements the OpenAI
Assistants runtime, and lets Conductor run the tools a hosted agent asks for as real workflow
tasks.
Supersedes #1446 and #1447. #1446 added an
AzureAgentRunStoreSPI to make the status pollstateless; this removes the need for a store at all, so that SPI is gone. #1447's auth modes,
agent discovery, AssumeRole, on-behalf-of hooks and UI are all folded in here — ported onto the
stateless client structure rather than the stateful one they were written against.
Hosted agents no longer keep per-run state
AzureFoundryAgentClientkeptendpoint,runIdand the token provider in an in-processConcurrentHashMap. A status poll routed to another replica hit "No execution found", so everyexecution after the first failed in a multi-replica deployment.
The fix is that there is nothing to remember. The
executionIdis the Azure thread id, and thethread is the conversation: the run to act on is always the newest one on it, which the provider
names on request. Everything else — endpoint, assistant, api version, credential, scope — is
re-derived from the task input Conductor already persists. A poll, a respond, or a cancel now
resolves identically on any replica, with no store to configure.
A status poll still costs one HTTP call: listing the newest run returns its full status, so
asking which run is current is no more expensive than having remembered its id.
OpenAI Assistants implemented; the protocol is now shared
Azure Foundry and OpenAI speak the same thread-and-run API, so that protocol moved into
AssistantsRunApiand both clients became thin layers over it — Azure adds Entra ID clientcredentials and
api-version, OpenAI adds an API key and theOpenAI-Betaheader.OpenAiAssistantsAgentClientwas a stub throwingUnsupportedOperationException; it is now aworking runtime.
Bedrock is stateless too, and no longer leaks
Bedrock has no status API —
InvokeAgentstreams the whole turn, so the agent has finished orblocked on a tool before the call returns. It buffered that result in a map, which meant a poll on
another replica reported a terminal failure for a run that had actually succeeded.
It now reports the outcome directly:
startAgentthroughConductorAgentStartResponse.state, andrespondthrough the new defaultrespondWithStatus, so the result lands in the task outputinstead of a map. Runtimes with a status API return null there and are polled exactly as before.
It also built one
BedrockAgentRuntimeAsyncClientper execution and never removed a finishedone, leaking Netty event loops and a connection pool per agent invocation. One client is now shared
per credential and region, released on shutdown.
Parallel tool calls no longer produce wrong answers
A model may ask for several independent tools in one turn. Only the first was reported, and the
reply was then written as the result for every outstanding call — a reply the provider accepts
and the model reasons from, so a second tool asking for headcount was told the revenue figure. Two
200s, a completed workflow, no error anywhere.
pendingToolsnow carries every call end to end, each is answered by its owntool_call_id, and areply that does not cover them all is rejected rather than padded. Single-tool turns are unchanged.
autoRunTools: the agent's tools become workflow tasksA function tool is one the platform cannot run — you registered only its schema. Previously the
AGENTtask completed withwaiting: trueand the workflow author hand-wired a dispatch branchand a resume task for every agent.
With
autoRunTools: truetheAGENTtask staysIN_PROGRESSwhile each requested tool isscheduled as an ordinary task named after the tool, so a worker already registered for
get_revenueserves it with no configuration. Tools fan out in parallel underFORK_JOIN_DYNAMIC, each with its own retries, timeout and execution history, and the agent isresumed with their results keyed by call id. Another turn asking for tools is simply another batch.
toolTaskNamesoverrides the naming convention.Off by default, so existing hand-wired workflows are untouched, and it degrades cleanly: a
remotely-polled SDK worker has no engine to schedule on, so the tool request is handed back as
before. When a tool exhausts its retries the remaining tools are stopped, the agent run is
cancelled, and the task fails with that tool's reason.
Token exchange per poll
OAuthTokenProvidercaches and refreshes a token, but a new provider was built on every call, soeach 5-second poll paid a full Entra ID round trip plus three secret-store reads. Providers are now
cached per credential and scope with a 10-minute TTL; a 401/403 evicts immediately so a rotated
credential is picked up on the next poll. A steady-state poll now performs no secret read at all.
Folded in from #1447
Four Azure auth modes, first match wins — API key (
api-keyheader, no SDK), service principal,user-assigned managed identity, and the default Azure credential chain. A deployment running on
managed identity now needs no
credentialRefat all. Resolved auth is cached per credential andscope, so a poll performs no secret-store read; a 401/403 evicts it immediately.
Running as the caller.
useCallerIdentity: trueexchanges the triggering user's Entra ID token,via the OAuth 2.0 on-behalf-of grant, for one scoped to Foundry — so the agent sees only what that
person can. Their own token never reaches Foundry, and the exchanged token is never cached, since it
belongs to a person rather than the deployment. Without an SSO-supplied assertion or a service
principal to perform the exchange, it falls back to credential auth rather than failing.
All three Foundry surfaces. Foundry is three APIs behind one
agentType: classic Assistants(threads and runs, pollable), a project's Responses API, and model inference. The latter two answer
inside the start call, so they report a terminal state through
ConductorAgentStartResponseinsteadof being polled — the same mechanism Bedrock uses, which makes them stateless with nothing to
remember. A project agent's own instructions and tools are forwarded so its web search, code
interpreter and file search run.
rawConfig.surfaceoverrides the hostname classification. #1447 classified purely by hostname, whichsilently misroutes sovereign clouds (
.azure.us,.azure.cn), private endpoints and proxies to theclassic path.
Agent discovery. A secret with an
endpointkey lists Azure agents; one with aregionkeylists Bedrock agents. They appear in the agent list beside agents defined in Conductor, with no
separate registration. Best effort — a credential that cannot list contributes nothing rather than
breaking the listing.
Bedrock AssumeRole (
roleArn,roleSessionName,externalId) alongside static keys and thedefault chain.
agentUrlas a top-level field for both providers, so every agent type names its location theway A2A does: Azure splits a trailing
/assistants/asst_xor/agents/NAMEoff as the agent,Bedrock parses
bedrock://AGENTID/ALIASID?region=.UI — provider logos and filter chips on the agent list, and a caller-identity toggle on the task
form, on top of the five-runtime typing already here.
Also in here
a2a-integration.mdsaida2awas "the only runtime in OSS today" and that anyother
agentType"is rejected today", while three provider runtimes were registered and working.Corrected, and a new Hosted Platform Agents page documents each runtime's
rawConfigkeys,credential shape, and the tool loop — written from the source.
AgentRuntimeTypewas"a2a" | "conductor", so aFoundry task was labelled an unresolved A2A agent with no name and triggered a doomed
/a2a/agent-carddiscovery call on save. All five runtimes are now typed and badged, hostedagents resolve without remote discovery, and the editor offers them with the right fields.
The execution view lists the tools an agent is waiting on and links to the run executing them.
content[0].textis empty for assistants with code interpreter) into theshared
AssistantsRunApi.extractText, with a regression test, so the rewrite does not lose it.agent_capabilities→tagsmapping, whichfeat(azure-foundry,bedrock): Entra ID auth modes, agent tools forwarding, and AssumeRole support #1447 dropped incidentally, and this branch's stateless execution model. Two things taken from
feat(azure-foundry,bedrock): Entra ID auth modes, agent tools forwarding, and AssumeRole support #1447 in preference to what was here: dropping
unresolvedfrom the task card (a workflowregistered outside the editor never has a snapshot, so it was never a real signal) and treating
the live input's
agentTypeas authoritative over a snapshot that lags a live edit.Alternatives considered
Compound
executionId(the approach in #1446's first commit) base64-encoded the run contextinto the id. Genuinely stateless, but the id is workflow-visible — the delegate mirrors it to
subWorkflowId— so endpoint and assistant leaked into the UI. Encoding only the thread id gets thesame property without that.
A durable
AzureAgentRunStore(#1446's second commit) added an SPI for hosts to implement withRedis or a DB. It shipped an in-process default, so OSS multi-replica stayed broken and every
embedder had infrastructure to supply. Since the provider is the source of truth for run state,
there was nothing that needed storing.
Tool tasks as inline children of the
AGENTtask, rather than a nested workflow, wasinvestigated and deferred. It needs no decider surgery, but the only thing it adds over this is
avoiding one extra workflow execution per tool turn — at the cost of new engine surface for
scheduling arbitrary runtime tasks, first use of the dormant
TaskModel.parentTaskId, and a UIconcept for a task owning children. Worth revisiting if per-turn executions prove costly.
Testing
:conductor-ai:testand:conductor-agentspan:testgreen, spotless cleancompileJava+compileTestJavaclean;mkdocs buildcleanui-next:tsc --noEmitandeslint --quietclean, 882 tests passEntra endpoint), Foundry surfaces and routing, OpenAI, Bedrock, the agent delegate, the tool
dispatcher, plus UI tests for runtime typing, the task form and the execution view
so an OkHttp interceptor no longer intercepts its token calls. Mode selection is asserted without
resolving a token
Not yet exercised against a live provider. The path worth a manual check before merge is an
autoRunToolsagent with two function tools: confirm both tool tasks are picked up in parallel,then terminate the parent mid-flight and confirm the tool workflow terminates with it.