Skip to content

Commit 2599e9b

Browse files
ericksoadeepujain
andcommitted
fix(e2e): match credential values in sandbox scan (#9395)
<!-- markdownlint-disable MD041 --> ## Summary The cloud inference credential scan treated npm lifecycle variable names in installed dependencies as credential leaks. It now matches the high-confidence provider formats owned by the security scanner and reports only matching file paths, so dependency metadata passes while credential canaries still fail safely. ## Related Issue Fixes #9363 ## Changes - Extract the live sandbox scan command into a focused helper so the exact production command is regression-tested. - Derive the in-process and POSIX sandbox patterns from one high-confidence provider/threshold table in the owning security module, including the underscore-bearing fine-grained GitHub PAT format. - Preserve the existing directory exclusions and grep error propagation while scanning text and NUL-containing files. - Cover the observed `npm_config_user_agent` and `$npm_package_version` dependency records, token-shaped dependency paths, payload and identifier boundaries, and redacted canaries for each credential family. - Consolidate the source-of-truth design from #9382 here with co-author credit to Deepak Jain. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Maintainer nine-category security review completed on the exact commit; no findings. The scan remains read-only, propagates errors, and emits paths rather than matched credential values. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh` is unchanged. - Station profile/scenario: Not applicable. - Result: Not applicable. - Supporting evidence: Not applicable. ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — exact shell/support suites (27/27 passed), secret-scanner suite (56/56 passed), and growth guardrails (22/22 passed) - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — CI pending; the local macOS run was inconclusive because unrelated environment-sensitive suites timed out or consumed ambient host state. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Security** - Improved credential-boundary scanning for cloud inference sandbox data. - Detects high-confidence NVIDIA, GitHub—including fine-grained—and npm credentials while redacting secret values. - Excludes policy, dependency, and benign metadata paths from findings. - Safely handles missing directories, embedded or short tokens, NUL-containing files, and expected no-match results. - Scan results identify only affected file paths, protecting credential contents. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> Co-authored-by: Deepak Jain <deepujain@gmail.com>
1 parent f08ae80 commit 2599e9b

5 files changed

Lines changed: 219 additions & 52 deletions

File tree

nemoclaw/src/security/secret-scanner.test.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ const FAKE = {
1111
openai: "sk-" + "abc123def456ghi789jkl012mno",
1212
openaiProject: "sk-proj-" + "abc123_def456-ghi789_jkl012-mno345",
1313
github: "ghp_" + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmn",
14+
githubFineGrained: "github_pat_" + "ABCDEFGHIJKLMNO_PQRSTUVWXYZabc",
1415
aws: "AKIA" + "IOSFODNN7EXAMPLE",
1516
slack: "xoxb-" + "123456789-abcdefghij",
1617
slackApp: "xapp-" + "1-A0000-12345-abcdef",
@@ -54,6 +55,12 @@ describe("scanForSecrets", () => {
5455
expect(matches[0].pattern).toBe("GitHub token");
5556
});
5657

58+
it("detects an underscore-bearing fine-grained GitHub personal access token", () => {
59+
const matches = scanForSecrets(`token: ${FAKE.githubFineGrained}`);
60+
expect(matches).toHaveLength(1);
61+
expect(matches[0].pattern).toBe("GitHub token");
62+
});
63+
5764
it("detects an AWS access key", () => {
5865
const matches = scanForSecrets(`aws_access_key_id = ${FAKE.aws}`);
5966
expect(matches).toHaveLength(1);

nemoclaw/src/security/secret-scanner.ts

Lines changed: 44 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,16 +21,55 @@ interface SecretPattern {
2121
regex: RegExp;
2222
}
2323

24+
/** Provider token formats shared by the in-process and sandbox scanners. */
25+
export const HIGH_CONFIDENCE_PREFIXED_TOKEN_SPECS = [
26+
{
27+
name: "NVIDIA API key",
28+
prefixes: ["nvapi-"],
29+
payloadCharacterClass: "A-Za-z0-9_-",
30+
minimumPayloadLength: 20,
31+
},
32+
{
33+
name: "GitHub token",
34+
prefixes: ["ghp_", "gho_", "ghu_", "ghs_", "ghr_"],
35+
payloadCharacterClass: "A-Za-z0-9",
36+
minimumPayloadLength: 36,
37+
},
38+
{
39+
name: "GitHub token",
40+
prefixes: ["github_pat_"],
41+
payloadCharacterClass: "A-Za-z0-9_",
42+
minimumPayloadLength: 30,
43+
},
44+
{
45+
name: "npm token",
46+
prefixes: ["npm_"],
47+
payloadCharacterClass: "A-Za-z0-9",
48+
minimumPayloadLength: 36,
49+
},
50+
] as const;
51+
52+
const HIGH_CONFIDENCE_PREFIXED_TOKEN_ALTERNATIVES = HIGH_CONFIDENCE_PREFIXED_TOKEN_SPECS.flatMap(
53+
({ prefixes, payloadCharacterClass, minimumPayloadLength }) =>
54+
prefixes.map((prefix) => `${prefix}[${payloadCharacterClass}]{${minimumPayloadLength},}`),
55+
).join("|");
56+
57+
/** POSIX ERE for standalone high-confidence provider tokens in sandbox shell scans. */
58+
export const HIGH_CONFIDENCE_PREFIXED_TOKEN_ERE = `(^|[^[:alnum:]_])(${HIGH_CONFIDENCE_PREFIXED_TOKEN_ALTERNATIVES})([^[:alnum:]_]|$)`;
59+
2460
const SECRET_PATTERNS: SecretPattern[] = [
25-
// NVIDIA
26-
{ name: "NVIDIA API key", regex: /\bnvapi-[A-Za-z0-9_-]{20,}\b/ },
61+
...HIGH_CONFIDENCE_PREFIXED_TOKEN_SPECS.map(
62+
({ name, prefixes, payloadCharacterClass, minimumPayloadLength }) => ({
63+
name,
64+
regex: new RegExp(
65+
`\\b(?:${prefixes.join("|")})[${payloadCharacterClass}]{${minimumPayloadLength},}\\b`,
66+
),
67+
}),
68+
),
2769

2870
// OpenAI — exclude sk-ant- (Anthropic) to avoid double-matching
2971
{ name: "OpenAI API key", regex: /\bsk-(?!ant-)[A-Za-z0-9_-]{20,}\b/ },
3072

31-
// GitHub
32-
{ name: "GitHub token", regex: /\b(ghp|gho|ghu|ghs|ghr|github_pat)_[A-Za-z0-9]{36,}\b/ },
33-
3473
// AWS
3574
{ name: "AWS access key", regex: /\bAKIA[0-9A-Z]{16}\b/ },
3675
{
@@ -48,9 +87,6 @@ const SECRET_PATTERNS: SecretPattern[] = [
4887
/(?<=(?:discord|bot|DISCORD_TOKEN|BOT_TOKEN|token)\s*[=:]\s*["']?)[A-Za-z0-9]{24}\.[A-Za-z0-9_-]{6}\.[A-Za-z0-9_-]{27,}/,
4988
},
5089

51-
// npm
52-
{ name: "npm token", regex: /\bnpm_[A-Za-z0-9]{36,}\b/ },
53-
5490
// Private keys (PEM)
5591
{
5692
name: "Private key",
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
// SPDX-License-Identifier: Apache-2.0
3+
4+
import { shellQuote } from "../../../src/lib/core/shell-quote.ts";
5+
import { HIGH_CONFIDENCE_PREFIXED_TOKEN_ERE } from "../../../nemoclaw/src/security/secret-scanner.ts";
6+
7+
const DEFAULT_SANDBOX_STATE_DIRECTORIES = ["/sandbox/.openclaw", "/sandbox/.nemoclaw"];
8+
9+
/** Build a path-only scan for concrete credential values in sandbox state. */
10+
export function buildSandboxCredentialScanCommand(
11+
directories: readonly string[] = DEFAULT_SANDBOX_STATE_DIRECTORIES,
12+
): string {
13+
const roots = directories.map((directory) => shellQuote(directory)).join(" ");
14+
return [
15+
`for dir in ${roots}; do`,
16+
' [ -d "$dir" ] || continue',
17+
` matches=$(grep -rlE '${HIGH_CONFIDENCE_PREFIXED_TOKEN_ERE}' "$dir")`,
18+
" scan_status=$?",
19+
' case "$scan_status" in',
20+
` 0) printf '%s\\n' "$matches" | grep -Ev '/policies/|/plugin-runtime-deps/|/extensions/[^/]+/(dist|node_modules)/'`,
21+
" filter_status=$?",
22+
' case "$filter_status" in',
23+
" 0|1) ;;",
24+
' *) exit "$filter_status" ;;',
25+
" esac",
26+
" ;;",
27+
" 1) ;;",
28+
' *) exit "$scan_status" ;;',
29+
" esac",
30+
"done",
31+
].join("\n");
32+
}

test/e2e/live/cloud-inference.test.ts

Lines changed: 2 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ import {
3232
parseCloudChatResponse,
3333
type PreContractExternalProviderFailure,
3434
} from "./cloud-inference-provider-skip.ts";
35+
import { buildSandboxCredentialScanCommand } from "./cloud-inference-credential-boundary.ts";
3536

3637
const REPO_SKILL_VALIDATOR = path.join(
3738
REPO_ROOT,
@@ -302,50 +303,7 @@ async function expectSandboxCredentialBoundary(
302303
"",
303304
);
304305

305-
const secretScanCommand = [
306-
"for dir in /sandbox/.openclaw /sandbox/.nemoclaw; do",
307-
' [ -d "$dir" ] || continue',
308-
` matches=$(grep -rIlE 'nvapi-|ghp_|npm_' "$dir")`,
309-
" scan_status=$?",
310-
' case "$scan_status" in',
311-
` 0) filtered=$(printf '%s\\n' "$matches" | grep -Ev '/policies/|/plugin-runtime-deps/|/extensions/[^/]+/(dist|node_modules)/')`,
312-
" filter_status=$?",
313-
' case "$filter_status" in',
314-
" 0) filtered_file=$(mktemp)",
315-
" temp_status=$?",
316-
' case "$temp_status" in 0) ;; *) exit "$temp_status" ;; esac',
317-
` trap 'rm -f "$filtered_file"' EXIT HUP INT TERM`,
318-
` printf '%s\\n' "$filtered" > "$filtered_file"`,
319-
" write_status=$?",
320-
' case "$write_status" in 0) ;; *) exit "$write_status" ;; esac',
321-
" while IFS= read -r file; do",
322-
` matching_lines=$(grep -IE 'nvapi-|ghp_|npm_' "$file")`,
323-
" match_status=$?",
324-
' case "$match_status" in',
325-
` 0) printf '%s' "$matching_lines" | grep -qv 'STRIPPED'`,
326-
" unstripped_status=$?",
327-
' case "$unstripped_status" in',
328-
` 0) printf '%s\\n' "$file" ;;`,
329-
" 1) ;;",
330-
' *) exit "$unstripped_status" ;;',
331-
" esac",
332-
" ;;",
333-
" 1) ;;",
334-
' *) exit "$match_status" ;;',
335-
" esac",
336-
' done < "$filtered_file"',
337-
' rm -f "$filtered_file"',
338-
" trap - EXIT HUP INT TERM",
339-
" ;;",
340-
" 1) ;;",
341-
' *) exit "$filter_status" ;;',
342-
" esac",
343-
" ;;",
344-
" 1) ;;",
345-
' *) exit "$scan_status" ;;',
346-
" esac",
347-
"done",
348-
].join("\n");
306+
const secretScanCommand = buildSandboxCredentialScanCommand();
349307

350308
const secretProbe = await sandbox.exec(SANDBOX_NAME, ["sh", "-lc", secretScanCommand], {
351309
artifactName: "phase-3-sandbox-secret-pattern-probe",
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
// SPDX-License-Identifier: Apache-2.0
3+
4+
import assert from "node:assert/strict";
5+
import { execFileSync } from "node:child_process";
6+
import fs from "node:fs";
7+
import os from "node:os";
8+
import path from "node:path";
9+
10+
import { afterEach, describe, expect, it } from "vitest";
11+
12+
import { HIGH_CONFIDENCE_PREFIXED_TOKEN_SPECS } from "../../../nemoclaw/src/security/secret-scanner.ts";
13+
import { buildSandboxCredentialScanCommand } from "../live/cloud-inference-credential-boundary.ts";
14+
15+
const roots: string[] = [];
16+
17+
afterEach(() => {
18+
for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true });
19+
});
20+
21+
/** Create and track an isolated sandbox-state fixture root. */
22+
function createScanRoot(): string {
23+
const root = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-cloud-credential-scan-"));
24+
roots.push(root);
25+
return root;
26+
}
27+
28+
/** Write one text or binary sandbox-state fixture and return its path. */
29+
function writeFixture(root: string, relativePath: string, body: string | Uint8Array): string {
30+
const rootPath = path.resolve(root);
31+
assert(!path.isAbsolute(relativePath), "Fixture path must be relative to the scan root");
32+
const file = path.resolve(rootPath, relativePath);
33+
const relativeFile = path.relative(rootPath, file);
34+
assert(
35+
relativeFile !== ".." && !relativeFile.startsWith(`..${path.sep}`),
36+
"Fixture path must stay inside the scan root",
37+
);
38+
fs.mkdirSync(path.dirname(file), { recursive: true });
39+
fs.writeFileSync(file, body);
40+
return file;
41+
}
42+
43+
/** Run the exact live credential scan command against a fixture root. */
44+
function scan(root: string): string {
45+
return execFileSync("sh", ["-lc", buildSandboxCredentialScanCommand([root])], {
46+
encoding: "utf8",
47+
});
48+
}
49+
50+
describe("cloud inference sandbox credential scan", () => {
51+
it("rejects fixture paths outside the temporary scan root", () => {
52+
const root = createScanRoot();
53+
54+
expect(() => writeFixture(root, "../outside.txt", "outside\n")).toThrow(
55+
"Fixture path must stay inside the scan root",
56+
);
57+
expect(() => writeFixture(root, path.join(root, "absolute.txt"), "outside\n")).toThrow(
58+
"Fixture path must be relative to the scan root",
59+
);
60+
});
61+
62+
it("accepts npm dependency metadata that does not contain a credential value (#9363)", () => {
63+
const root = createScanRoot();
64+
writeFixture(
65+
root,
66+
"npm/projects/openclaw-whatsapp/node_modules/thread-stream/test/ts/transpile.sh",
67+
'echo "${npm_config_user_agent}"\n',
68+
);
69+
writeFixture(
70+
root,
71+
"npm/projects/openclaw-msteams/node_modules/jwks-rsa/package.json",
72+
'{"scripts":{"release":"git tag $npm_package_version"}}\n',
73+
);
74+
writeFixture(root, "configuration/token-key-path.txt", "ordinary dependency metadata\n");
75+
76+
expect(scan(root)).toBe("");
77+
});
78+
79+
it.each([
80+
["NVIDIA", "nvapi-nemoclaw-credential-boundary-canary"],
81+
["GitHub", `ghp_${"a".repeat(36)}`],
82+
["GitHub fine-grained", `github_pat_${"a".repeat(15)}_${"b".repeat(14)}`],
83+
["npm", `npm_${"b".repeat(36)}`],
84+
])("reports only the path of a file that contains a %s credential canary", (_label, canary) => {
85+
const root = createScanRoot();
86+
const leakedFile = writeFixture(root, "openclaw.json", `{"apiKey":"${canary}"}\n`);
87+
88+
const output = scan(root);
89+
90+
expect(output.trim()).toBe(leakedFile);
91+
expect(output).not.toContain(canary);
92+
});
93+
94+
it.each(
95+
HIGH_CONFIDENCE_PREFIXED_TOKEN_SPECS.flatMap(({ prefixes, minimumPayloadLength }) =>
96+
prefixes.map((prefix) => [prefix, minimumPayloadLength] as const),
97+
),
98+
)("enforces the shared minimum payload for %s", (prefix, minimumPayloadLength) => {
99+
const root = createScanRoot();
100+
writeFixture(root, "short.txt", `${prefix}${"a".repeat(minimumPayloadLength - 1)}\n`);
101+
102+
expect(scan(root)).toBe("");
103+
104+
const detectedFile = writeFixture(
105+
root,
106+
"minimum.txt",
107+
`${prefix}${"a".repeat(minimumPayloadLength)}\n`,
108+
);
109+
expect(scan(root).trim()).toBe(detectedFile);
110+
});
111+
112+
it.each([
113+
["prefixed GitHub token", `prefixghp_${"a".repeat(36)}`],
114+
["suffixed GitHub token", `ghp_${"a".repeat(36)}_suffix`],
115+
["prefixed npm token", `prefixnpm_${"b".repeat(36)}`],
116+
["suffixed npm token", `npm_${"b".repeat(36)}_suffix`],
117+
])("does not report a token embedded in a larger identifier: %s", (_label, value) => {
118+
const root = createScanRoot();
119+
writeFixture(root, "embedded.txt", `${value}\n`);
120+
121+
expect(scan(root)).toBe("");
122+
});
123+
124+
it("reports a credential canary in a NUL-containing file", () => {
125+
const root = createScanRoot();
126+
const canary = "nvapi-nemoclaw-binary-credential-canary";
127+
const leakedFile = writeFixture(root, "state.bin", Buffer.from(`prefix\0${canary}\n`));
128+
129+
const output = scan(root);
130+
131+
expect(output.trim()).toBe(leakedFile);
132+
expect(output).not.toContain(canary);
133+
});
134+
});

0 commit comments

Comments
 (0)