Security: Require user approval before agent executes toolset calls (human-in-the-loop gate)

[photonn]

Description

Problem

When a DIAL agent is configured with toolsets (e.g., database access, file system, external APIs), it can call those tools fully autonomously — including in response to malicious instructions delivered via indirect prompt injection. There is currently no mechanism to pause execution and request explicit user approval before a tool call is made.

This creates a critical risk when an agent has what can be described as the "three dangerous legs" simultaneously:

1. Fully autonomous execution — the agent calls tools without human confirmation
2. Access to the internet — the agent can reach external URLs (via web search, fetch, or other toolsets)
3. Access to internal/sensitive data — the agent can read databases, documents, or internal APIs
   Any attacker who can place malicious content in data the agent reads (a document, a web page, a database row) can hijack the agent's tool-calling behavior, causing it to exfiltrate, modify, or delete data without any user awareness.

Proposed solution

Add a configurable human-in-the-loop approval gate for toolset calls, similar to the security model implemented in popular AI coding assistants (e.g., Claude Code, Cursor, Windsurf):

• Per-toolset approval mode: each toolset can be configured as auto (no approval needed) or require-approval (user must confirm before each call).
• Approval UI: when the agent wants to call a tool, display a modal/inline prompt showing:
• • Tool name and toolset
• • Parameters the agent wants to pass
• • Approve / Deny buttons
• • Session-level trust: optionally allow the user to approve a toolset "for this session" to reduce friction for trusted workflows.
• • Audit log: all tool calls (approved or denied) are logged so administrators can review agent behavior.

Prior art / references

• Claude Code: prompts the user before executing bash commands, writing files, or calling external APIs
• Cursor / Windsurf: "agent mode" shows tool calls and asks for confirmation on destructive or network operations
• OWASP LLM Top 10 – LLM06: Excessive Agency (recommends human oversight for consequential agent actions)

Use case/motivation

In enterprise environments, DIAL is deployed with agents that have toolsets connecting to internal systems (databases, HR systems, internal APIs, document stores). These agents may also browse the web or process external documents.

Without an approval gate, a single successful prompt injection — for example, a malicious instruction hidden in a PDF the agent is asked to summarize — can cause the agent to autonomously call any tool it has access to, with arbitrary parameters, without the user ever knowing.

The combination of autonomous execution + internet access + internal data access is the most dangerous configuration for any AI agent. Popular developer tools have already recognized this and implemented human-in-the-loop gates. DIAL should offer the same protection for enterprise chat users who may be less aware of these risks than developers.

Confidential information

• I confirm that do not share any confidential information

[rpelevin]

The per-toolset approval mode makes sense, but I would make the approval bind to the exact proposed tool-call envelope, not only to the toolset name shown in the UI.

For enterprise chat, the risky case is usually not "this toolset is always bad"; it is "this exact call with these parameters, resource, user context, and policy state is consequential."

A practical approval record could include:

• conversation/run id;
• agent id;
• toolset and tool name;
• target resource or tenant scope;
• canonical args/request digest;
• risk class;
• approval mode: per-call, session-scoped, denied, expired;
• decision id;
• approver identity class;
• issued_at / expires_at / consumed_at;
• policy version or rule id.

Session-level trust should also stay scoped. "Trust this toolset for this session" should not silently authorize a different resource, a higher-risk tool, changed args, or a later policy version.

Regression tests I would add:

1. require-approval pauses before the tool handler receives the call.
2. approving resumes exactly the displayed call once.
3. deny, timeout, or closing the prompt produces a terminal non-execution result.
4. changed args, tool name, resource, user, or policy version requires a new approval.
5. session trust is scoped by toolset/tool, risk tier, resource class, and expiry.
6. audit logs store the decision id and digest without requiring raw sensitive args.

That keeps the approval gate useful against indirect prompt injection: the model can propose the call, but the runtime still decides whether that exact call is allowed to execute.