Skip to content

feat(safety): add native PII detection and response validation to generate/stream #952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
murdore wants to merge 1 commit into
base: release
Choose a base branch
from
Open

feat(safety): add native PII detection and response validation to generate/stream #952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions src/cli/factories/commandFactory.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -484,6 +484,56 @@ export class CLICommandFactory {
alias: "rag-top-k",
default: 5,
},
// Safety — PII detection
piiRedact: {
type: "boolean" as const,
description:
"Enable PII detection and redaction on input before sending to LLM",
alias: "pii-redact",
default: false,
},
piiTypes: {
type: "string" as const,
description:
"Comma-separated PII types to detect (email,phone,ssn,creditCard,ipAddress,address,name,dateOfBirth,passport,driversLicense)",
alias: "pii-types",
},
piiAction: {
type: "string" as const,
description: "Action when PII is found: redact, abort, or warn",
alias: "pii-action",
default: "redact",
},
// Safety — Input validation
inputMaxLength: {
type: "number" as const,
description: "Maximum input text length (characters)",
alias: "input-max-length",
},
trimWhitespace: {
type: "boolean" as const,
description: "Trim whitespace from input text",
alias: "trim-whitespace",
default: false,
},
requireContent: {
type: "boolean" as const,
description: "Abort if input text is empty or whitespace",
alias: "require-content",
default: false,
},
// Safety — Response validation
outputMaxLength: {
type: "number" as const,
description:
"Maximum response length (characters). Truncates if exceeded",
alias: "output-max-length",
},
outputMinLength: {
type: "number" as const,
description: "Minimum response length (characters)",
alias: "output-min-length",
},
Comment on lines +487 to +536

@coderabbitai coderabbitai Bot Apr 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Avoid exposing safety flags on commands that silently ignore them.

Line 487 adds these flags to commonOptions, so batch, provider status, setup, loop, and other commands accept them too. Only executeGenerate() and executeRealStream() forward them; for example, neurolink batch prompts.txt --pii-redact is accepted but executeBatch() does not pass piiDetection, so prompts can be sent unredacted. Move these flags into generate/stream-specific options, or wire them through every command that exposes them.

Suggested direction 
+  private static readonly safetyOptions = {
+    // Safety — PII detection
+    piiRedact: { /* existing option */ },
+    piiTypes: { /* existing option */ },
+    piiAction: { /* existing option */ },
+    // Safety — Input validation
+    inputMaxLength: { /* existing option */ },
+    trimWhitespace: { /* existing option */ },
+    requireContent: { /* existing option */ },
+    // Safety — Response validation
+    outputMaxLength: { /* existing option */ },
+    outputMinLength: { /* existing option */ },
+  };
+
   private static readonly commonOptions = {
     // Core generation options
     provider: {
       ...
     },
-    // Safety — PII detection
-    piiRedact: { ... },
-    piiTypes: { ... },
-    piiAction: { ... },
-    // Safety — Input validation
-    inputMaxLength: { ... },
-    trimWhitespace: { ... },
-    requireContent: { ... },
-    // Safety — Response validation
-    outputMaxLength: { ... },
-    outputMinLength: { ... },
   };

Then pass CLICommandFactory.safetyOptions as additionalOptions only from createGenerateCommand() and createStreamCommand() unless batch support is intentionally implemented too.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@src/cli/factories/commandFactory.ts` around lines 487 - 536, commonOptions
currently includes safety flags (piiRedact, piiTypes, piiAction, inputMaxLength,
trimWhitespace, requireContent, outputMaxLength, outputMinLength) which makes
commands like executeBatch() accept them but not apply them; either move these
flags out of commonOptions into the generate/stream-specific option set or
ensure every command that exposes them forwards them to the execution layer.
Specifically, remove or stop exporting the safety flags from commonOptions (or
CLICommandFactory.commonOptions) and instead add CLICommandFactory.safetyOptions
only to createGenerateCommand() and createStreamCommand(); alternatively, if you
prefer global exposure, update executeBatch(), executeProviderStatus(),
executeSetup(), executeLoop(), etc. to accept and pass the corresponding
piiDetection/input validation/response validation fields through to the same
place executeGenerate() and executeRealStream() use so the flags are honored.

};

// Helper method to build options for commands
Expand Down Expand Up @@ -2662,6 +2712,48 @@ export class CLICommandFactory {
topK: argv.ragTopK as number | undefined,
}
: undefined,
// PII detection
piiDetection: argv.piiRedact
? {
enabled: true,
action:
(argv.piiAction as "redact" | "abort" | "warn") ?? "redact",
detectTypes: argv.piiTypes
? ((argv.piiTypes as string)
.split(",")
.map((t) => t.trim()) as Array<
| "email"
| "phone"
| "ssn"
| "creditCard"
| "ipAddress"
| "address"
| "name"
| "dateOfBirth"
| "passport"
| "driversLicense"
>)
: undefined,
}
: undefined,
// Input validation
inputValidation:
argv.inputMaxLength || argv.trimWhitespace || argv.requireContent
? {
maxLength: argv.inputMaxLength as number | undefined,
trimWhitespace: argv.trimWhitespace as boolean | undefined,
requireContent: argv.requireContent as boolean | undefined,
}
: undefined,
// Response validation
responseValidation:
argv.outputMaxLength || argv.outputMinLength
? {
maxLength: argv.outputMaxLength as number | undefined,
minLength: argv.outputMinLength as number | undefined,
truncationAction: "truncate" as const,
}
: undefined,
// TTS configuration
tts: enhancedOptions.tts
? {
Expand Down Expand Up @@ -2942,6 +3034,48 @@ export class CLICommandFactory {
topK: argv.ragTopK as number | undefined,
}
: undefined,
// PII detection
piiDetection: argv.piiRedact
? {
enabled: true,
action:
(argv.piiAction as "redact" | "abort" | "warn") ?? "redact",
detectTypes: argv.piiTypes
? ((argv.piiTypes as string)
.split(",")
.map((t) => t.trim()) as Array<
| "email"
| "phone"
| "ssn"
| "creditCard"
| "ipAddress"
| "address"
| "name"
| "dateOfBirth"
| "passport"
| "driversLicense"
>)
: undefined,
}
: undefined,
// Input validation
inputValidation:
argv.inputMaxLength || argv.trimWhitespace || argv.requireContent
? {
maxLength: argv.inputMaxLength as number | undefined,
trimWhitespace: argv.trimWhitespace as boolean | undefined,
requireContent: argv.requireContent as boolean | undefined,
}
: undefined,
// Response validation
responseValidation:
argv.outputMaxLength || argv.outputMinLength
? {
maxLength: argv.outputMaxLength as number | undefined,
minLength: argv.outputMinLength as number | undefined,
truncationAction: "truncate" as const,
}
: undefined,
// TTS configuration
tts: enhancedOptions.tts
? {
Expand Down
4 changes: 4 additions & 0 deletions src/cli/loop/optionsSchema.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ export const textGenerationOptionsSchema: Record<
| "toolChoice" // Complex type, not suitable for simple CLI input
| "prepareStep" // Callback function, only usable via SDK
| "credentials" // Complex per-provider object, only usable via SDK
| "processors" // Complex I/O processor config, only usable via SDK
| "piiDetection" // Complex config, wired via CLI flags instead
| "responseValidation" // Complex config, wired via CLI flags instead
| "inputValidation" // Complex config, wired via CLI flags instead
>,
OptionSchema
> = {
Expand Down
12 changes: 12 additions & 0 deletions src/lib/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1150,3 +1150,15 @@ export {
// Server Bridge
createAuthValidatorFromProvider,
} from "./auth/index.js";

// ============================================================================
// SAFETY UTILITIES — PII Detection, Response Validation, Tripwires
// ============================================================================

export { detectAndRedactPII } from "./utils/piiDetector.js";
export { validateResponse } from "./utils/responseValidator.js";
export {
TripwireEvaluator,
createDefaultTripwireEvaluator,
commonTripwires,
} from "./utils/tripwireEvaluator.js";
115 changes: 115 additions & 0 deletions src/lib/neurolink.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,8 @@
import { isNonNullObject } from "./utils/typeUtils.js";
import { getWorkflow } from "./workflow/core/workflowRegistry.js";
import { runWorkflow } from "./workflow/core/workflowRunner.js";
import { detectAndRedactPII } from "./utils/piiDetector.js";
import { validateResponse } from "./utils/responseValidator.js";

/**
* NL-002: Classify MCP error messages into categories for AI disambiguation.
Expand Down Expand Up @@ -3179,7 +3181,7 @@
* Initialize event listeners that feed span data to MetricsAggregator.
* Listens to generation:end, stream:complete, and tool:end events.
*/
private initializeMetricsListeners(): void {

Check warning on line 3184 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / 🛡️ Code Quality & Security Gate 

Method 'initializeMetricsListeners' has too many lines (334). Maximum allowed is 300

Check warning on line 3184 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / test (20) 

Method 'initializeMetricsListeners' has too many lines (334). Maximum allowed is 300

Check warning on line 3184 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / test (20) 

Method 'initializeMetricsListeners' has too many lines (334). Maximum allowed is 300
this.emitter.on("generation:end", ((...args: unknown[]) => {
const data = args[0] as Record<string, unknown>;
// A2 fix: When Pipeline A (AI SDK → @langfuse/otel) already creates a
Expand Down Expand Up @@ -3896,6 +3898,49 @@
options.input?.text,
"Input text is required and must be a non-empty string",
);

// Input validation (trimWhitespace, minLength, maxLength, requireContent)
if (options.inputValidation) {
const iv = options.inputValidation;
if (iv.trimWhitespace) {
options.input.text = options.input.text.trim();
}
if (iv.requireContent && !options.input.text.trim()) {
throw new Error(
"Input content is required but was empty or whitespace",
);
}
if (iv.minLength && options.input.text.length < iv.minLength) {
throw new Error(
`Input text is too short (${options.input.text.length} < ${iv.minLength})`,
);
}
if (iv.maxLength && options.input.text.length > iv.maxLength) {
throw new Error(
`Input text is too long (${options.input.text.length} > ${iv.maxLength})`,
);
}
}

// PII detection and redaction
if (options.piiDetection?.enabled) {
const piiResult = await detectAndRedactPII(options.input.text, {
enabled: true,
action: options.piiDetection.action ?? "warn",
detectTypes: options.piiDetection.detectTypes,
customPatterns: options.piiDetection.customPatterns,
allowList: options.piiDetection.allowList,
redactionText: options.piiDetection.redactionText,
});
if (piiResult.action === "abort") {
throw new Error(
piiResult.feedback ?? "Request blocked: PII detected in input",
);
Comment on lines +3908 to +3938

@coderabbitai coderabbitai Bot Apr 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Return typed validation errors for the new guardrails.

These new validation/PII/response-validation failures currently throw bare Error, so SDK callers cannot reliably distinguish validation failures, PII blocks, and retryable provider failures. Prefer an ErrorFactory validation/safety helper, or add one if none exists. As per coding guidelines, **/*.ts: Error Handling — Use ErrorFactory for typed errors, wrap async calls with withTimeout utility, formatProviderError must return errors never throw.

Also applies to: 4002-4006, 6419-6449

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@src/lib/neurolink.ts` around lines 3619 - 3649, Replace bare throws in the
guardrail/PII/validation paths with typed errors created via the project's
ErrorFactory (e.g., use ErrorFactory.createValidationError / createSafetyError)
instead of `throw new Error(...)` for the checks inside the input validation
block and the PII block that calls detectAndRedactPII; wrap the async
detectAndRedactPII call with the withTimeout utility and propagate its typed
error rather than raw throws (use the returned ErrorFactory error when
piiResult.action === "abort"); ensure any provider error formatting uses
formatProviderError to produce an Error object (never throw inside
formatProviderError) so SDK callers can reliably detect validation vs retryable
provider failures.

}
// Replace input text with redacted version
options.input.text = piiResult.text;
}

this.enforceSessionBudget(options.maxBudgetUsd);
this.applyGenerateLifecycleMiddleware(options);
await this.applyAuthenticatedRequestContext(options);
Expand Down Expand Up @@ -4229,6 +4274,34 @@
...(textResult.retries && { retries: textResult.retries }),
};

// Response validation (if configured)
if (options.responseValidation) {
const validationResult = validateResponse(generateResult.content ?? "", {
minLength: options.responseValidation.minLength,
maxLength: options.responseValidation.maxLength,
requiredPhrases: options.responseValidation.requiredPhrases,
forbiddenPhrases: options.responseValidation.forbiddenPhrases,
jsonSchema: options.responseValidation.jsonSchema,
customValidator: options.responseValidation.customValidator,
truncationAction: options.responseValidation.truncationAction,
truncationSuffix: options.responseValidation.truncationSuffix,
retryOnFailure: options.responseValidation.retryOnFailure,
maxRetries: options.responseValidation.maxRetries,
});

if (validationResult.action === "abort") {
throw new Error(
validationResult.feedback ??
`Response validation failed: ${validationResult.issues.map((i) => i.message).join("; ")}`,
);
}

// Apply validated/truncated text
if (validationResult.text !== generateResult.content) {
generateResult.content = validationResult.text;
}
}

if (generateResult.analytics?.cost && generateResult.analytics.cost > 0) {
this._sessionCostUsd += generateResult.analytics.cost;
}
Expand Down Expand Up @@ -6138,7 +6211,7 @@
/**
* Direct provider generation (no MCP, no recursion)
*/
private async directProviderGeneration(

Check warning on line 6214 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / 🛡️ Code Quality & Security Gate 

Async method 'directProviderGeneration' has too many lines (408). Maximum allowed is 300

Check warning on line 6214 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / test (20) 

Async method 'directProviderGeneration' has too many lines (408). Maximum allowed is 300

Check warning on line 6214 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / test (20) 

Async method 'directProviderGeneration' has too many lines (408). Maximum allowed is 300
options: TextGenerationOptions,
): Promise<TextGenerationResult> {
const startTime = Date.now();
Expand Down Expand Up @@ -6999,6 +7072,48 @@
startTime: number,
): Promise<void> {
await this.validateStreamInput(options);

// Input validation for stream
if (options.inputValidation && options.input?.text) {
const iv = options.inputValidation;
if (iv.trimWhitespace) {
options.input.text = options.input.text.trim();
}
if (iv.requireContent && !options.input.text.trim()) {
throw new Error(
"Input content is required but was empty or whitespace",
);
}
if (iv.minLength && options.input.text.length < iv.minLength) {
throw new Error(
`Input text is too short (${options.input.text.length} < ${iv.minLength})`,
);
}
if (iv.maxLength && options.input.text.length > iv.maxLength) {
throw new Error(
`Input text is too long (${options.input.text.length} > ${iv.maxLength})`,
);
}
}

// PII detection and redaction for stream
if (options.piiDetection?.enabled && options.input?.text) {
const piiResult = await detectAndRedactPII(options.input.text, {
enabled: true,
action: options.piiDetection.action ?? "warn",
detectTypes: options.piiDetection.detectTypes,
customPatterns: options.piiDetection.customPatterns,
allowList: options.piiDetection.allowList,
redactionText: options.piiDetection.redactionText,
});
if (piiResult.action === "abort") {
throw new Error(
piiResult.feedback ?? "Request blocked: PII detected in input",
);
}
options.input.text = piiResult.text;
}
Comment on lines +7076 to +7115

@coderabbitai coderabbitai Bot Apr 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Apply output response validation in the streaming path too.

The stream path now has input-side safety, but streamed output never calls validateResponse(): raw chunks are yielded and stream:complete is emitted with unvalidated accumulatedContent. This makes responseValidation a no-op for stream().

For streaming, enforce maxLength/truncation while yielding, and apply abort rules before stream:complete, span finalization, and memory storage.

Also applies to: 6584-6661

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@src/lib/neurolink.ts` around lines 6413 - 6452, The streaming path currently
validates input but never validates streamed output: update the stream()
implementation to call validateResponse() (or the existing responseValidation
logic) on the accumulatedContent before emitting the final "stream:complete" and
before persisting to memory; while streaming, enforce maxLength by truncating
yielded content when necessary and stop/abort if responseValidation returns an
abort rule (ensure validateResponse/responseValidation is invoked after each
chunk or at least on the final accumulated response), apply truncation to
accumulatedContent used for span finalization and memory storage, and ensure any
abort reason from validateResponse is thrown so spans are finalized and no
unvalidated content is stored (refer to function/method names validateResponse,
stream(), accumulatedContent, responseValidation, and the "stream:complete"
emission).


this.enforceSessionBudget(options.maxBudgetUsd);
await this.applyAuthenticatedRequestContext(options);
this.emitStreamStartEvents(options, startTime);
Expand Down Expand Up @@ -7046,7 +7161,7 @@
return result;
}

private async runStandardStreamRequest(params: {

Check warning on line 7164 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / 🛡️ Code Quality & Security Gate 

Async method 'runStandardStreamRequest' has too many lines (427). Maximum allowed is 300

Check warning on line 7164 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / test (20) 

Async method 'runStandardStreamRequest' has too many lines (427). Maximum allowed is 300

Check warning on line 7164 in src/lib/neurolink.ts

View workflow job for this annotation

GitHub Actions / test (20) 

Async method 'runStandardStreamRequest' has too many lines (427). Maximum allowed is 300
options: StreamOptions;
streamSpan: ReturnType<typeof tracers.sdk.startSpan>;
spanStartTime: number;
Expand Down
10 changes: 10 additions & 0 deletions src/lib/processors/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,3 +334,13 @@ export {
loadFileFromPath,
processFileFromPath,
} from "./cli/index.js";

// =============================================================================
// STANDALONE SAFETY UTILITIES (extracted from the former I/O processor system)
// =============================================================================

// PII detection, response validation, and tripwire evaluation are now
// standalone utilities in src/lib/utils/ and wired directly into
// generate() and stream() via native options (piiDetection, responseValidation,
// inputValidation). See src/lib/utils/piiDetector.ts, responseValidator.ts,
// and tripwireEvaluator.ts.
Loading
Loading