You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Target deployment: AWS on-prem architecture (Fargate agents, EC2 Nitro server). Architecture should be cloud-agnostic via adapters — AWS is first implementation.
6
+
7
+
All explicit threshold values, file paths, and integration details are preliminary and need to be validated against production data. Review after first month of real usage.
6
8
7
9
## Threat Categories
8
10
@@ -18,7 +20,7 @@ Detect adversarial instructions injected into the LLM context from untrusted sou
18
20
19
21
**Detection:** Arthur Engine Prompt Injection evaluation (DeBERTa v3 classifier).
@@ -28,14 +30,16 @@ Detect unauthorized outbound communication from agent sandboxes. The primary fea
28
30
29
31
- Outbound connections — every TCP connection from Fargate agents via VPC Flow Logs on `sg-fargate`
30
32
- DNS queries — domain names the agent resolves via VPC DNS query logging
31
-
- DNS tunneling — detect data exfiltration via DNS by monitoring for unusually long subdomain labels (>50 chars), high query volume to a single domain (>100 queries/min), and TXT record queries to non-standard domains
32
33
- Traffic volume — bytes uploaded per connection via VPC Flow Logs aggregation
33
-
- Unauthorized endpoints — connections to IPs/domains outside GitHub + Anthropic API
34
34
- Large uploads — unusual outbound data volume (e.g., >10MB to a single IP)
No endpoint allowlist — the agent needs unrestricted research access (docs, Stack Overflow, npm registries, etc.). Restricting outbound destinations would degrade research quality. Instead, rely on traffic volume anomaly detection and egress proxy logging to catch exfiltration after the fact.
37
+
38
+
DNS tunneling detection (subdomain length / query volume heuristics) was evaluated and removed — too spotty to be reliable, a determined attacker would trivially bypass the thresholds. Better to invest in egress proxy with TLS inspection (future).
39
+
40
+
**Detection:** AWS-native — VPC Flow Logs, DNS query logging, CloudWatch metric filters. Architecture should be cloud-agnostic via an adapter — AWS is first implementation, but GCP (VPC Flow Logs equivalent) and Azure (NSG Flow Logs) will need their own adapters.
**Response:**Volume anomaly → medium (flag + Slack alert). For true prevention (not just detection), the long-term approach is an egress proxy that blocks uploads above a size threshold before the connection completes — killing a sandbox after exfiltration already happened is pointless.
39
43
40
44
### 3. Secrets & Credential Leakage
41
45
@@ -50,7 +54,7 @@ Prevent API keys, tokens, passwords, and connection strings from appearing in ge
**Response:** Critical — no PR created, sandbox killed, ticket to "Security Review", Slack alert with redacted details.
57
+
**Response:** Critical — no PR created, sandbox killed, Slack alert with redacted details.
54
58
55
59
### 4. PII & Sensitive Business Data
56
60
@@ -64,25 +68,9 @@ Detect personally identifiable information and confidential business data in inp
64
68
65
69
**Detection:** Arthur Engine PII Detection (Presidio) for standard PII (names, emails, SSNs, credit cards). Arthur Engine Sensitive Data evaluation (few-shot LLM Judge) for business-specific confidential data — requires custom examples defining what "sensitive" means for the organization.
66
70
67
-
**Response:** Critical PII (SSN, credit card) → high (block PR). Other PII/sensitive data → medium (PR with `security-review` label + comment).
68
-
69
-
### 5. Code Safety & Vulnerability Detection
70
-
71
-
Detect common security vulnerabilities in LLM-generated code.
72
-
73
-
**Checks:**
74
-
75
-
- SQL injection patterns in generated database queries
76
-
- Command injection via shell commands, exec calls
77
-
- XSS vulnerabilities from unsanitized user input rendering
78
-
- Insecure dependencies — newly added packages with known CVEs
79
-
- Hardcoded secrets (overlaps with threat 3)
80
-
81
-
**Detection:** Arthur Engine custom rules (keyword + regex) for OWASP Top 10 patterns. Potential future integration with a dedicated SAST tool for deeper analysis.
82
-
83
-
**Response:** High — block PR creation, move ticket to "Security Review", Slack alert.
71
+
**Response:** Critical PII (SSN, credit card) → high (block PR, Slack alert). Other PII/sensitive data → medium (PR with `security-review` label + comment, Slack notification).
84
72
85
-
### 6. Behavioral Anomalies
73
+
### 5. Behavioral Anomalies
86
74
87
75
Detect unusual agent behavior that may indicate compromise or malfunction.
88
76
@@ -108,7 +96,7 @@ Detect unusual agent behavior that may indicate compromise or malfunction.
108
96
109
97
Thresholds are static at launch. After 30 days of production data, revisit and consider adaptive baselines derived from rolling 7-day percentiles (p95).
110
98
111
-
**Response:**Low — log only, included in Slack usage report.
99
+
**Response:**Token budget exceeded → critical (kill sandbox — circuit breaker for the infinite loop P0 bug that burned 100M+ tokens). Token usage must have a hard ceiling per phase, not a soft alert. All other anomalies → low (log only, included in Slack usage report).
112
100
113
101
## Pipeline Integration
114
102
@@ -131,7 +119,6 @@ Jira Ticket Discovered
131
119
| |
132
120
| Runtime monitoring: |
133
121
| - VPC Flow Logs (network) |
134
-
| - DNS query logging |
135
122
| - CloudWatch (tool usage) |
136
123
| |
137
124
| WebFetch interception: |
@@ -158,7 +145,6 @@ Jira Ticket Discovered
158
145
+-----------------------------+
159
146
| OUTPUT GATE (post-impl) |
160
147
| - All output checks |
161
-
| - Code safety (OWASP) |
162
148
| - Secrets in generated code |
163
149
+-------------+---------------+
164
150
| pass
@@ -185,7 +171,6 @@ Jira Ticket Discovered
185
171
| - Final secrets scan on |
186
172
| full PR diff |
187
173
| - Final PII check |
188
-
| - Code safety (OWASP) |
189
174
+-------------+---------------+
190
175
| pass
191
176
v
@@ -197,7 +182,7 @@ Jira Ticket Discovered
197
182
198
183
- Input gate — in `agentWorkflow` before `writeAndStartPhase`
199
184
- WebFetch interception — hook/proxy inside the agent container
- Output gate — in `collectPhaseOutput` before returning results
202
187
- Pre-push gate — in `pushFromSandbox` before the git push
203
188
- Fix-and-retry path — `fixAndRetryPush` in `poll-agent.ts` spawns a lightweight Claude agent to fix push failures. This agent receives untrusted input (the push error) and runs with `--dangerously-skip-permissions`. Its output must pass through the output gate and pre-push gate before the retry push proceeds.
0 commit comments