Fix for Imperva Cloud WAF CCF (S3‑Generic) Connector silently dropping logs which containing quotation characters#14037
Closed
v-krishnachi wants to merge 2 commits intoAzure:masterfrom
Closed
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Disables RFC 4180 quote-boundary handling for the Imperva Cloud WAF CCF (S3-Generic) connector to prevent log lines containing embedded JSON quotes from being dropped, and bumps the solution version accordingly.
Changes:
- Added
HasCsvBoundary: falseto CSVdataFormatin the connector’s polling and deployment templates. - Bumped solution version to 3.1.1 across solution metadata/templates.
- Added a 3.1.1 entry to release notes describing the fix.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| Solutions/ImpervaCloudWAF/ReleaseNotes.md | Adds release note for 3.1.1 documenting the CSV boundary handling fix. |
| Solutions/ImpervaCloudWAF/Package/mainTemplate.json | Bumps template/content versions and adds HasCsvBoundary: false in data format configuration. |
| Solutions/ImpervaCloudWAF/Data/Solution_ImpervaCloudWAF.json | Bumps solution version to 3.1.1. |
| Solutions/ImpervaCloudWAF/Data Connectors/ImpervaCloudWAFLogs_ccf/ImpervaCloudWAFLogs_PollingConfig.json | Adds HasCsvBoundary: false to polling config data format. |
| "Metadata": "SolutionMetadata.json", | ||
| "Version": "3.1.0", | ||
| "Version": "3.1.1", | ||
| "TemplateSpec": true, |
There was a problem hiding this comment.
TemplateSpec must be set to false for Version 3.. solutions; leaving it true will violate solution-data validation rules and can break packaging/validation. Update to "TemplateSpec": false for 3.1.1.
Suggested change
| "TemplateSpec": true, | |
| "TemplateSpec": false, |
| @@ -1,5 +1,6 @@ | |||
| | **Version** | **Date Modified (DD-MM-YYYY)** | **Change History** | | |||
| |-------------|--------------------------------|------------------------------------------------- | | |||
| | 3.1.1 | 09-04-2026 | Fixed CCF connector silently dropping logs containing quotation characters in JSON fields (e.g. cs10, cs11, additionalReqHeaders) by disabling RFC 4180 CSV boundary handling | | |||
There was a problem hiding this comment.
The Change History entry is unusually long for the release notes table and includes extra leading whitespace. Please shorten it to 1–2 sentences (and trim leading spaces) to keep the table readable and consistent with the release notes formatting guidance.
Suggested change
| | 3.1.1 | 09-04-2026 | Fixed CCF connector silently dropping logs containing quotation characters in JSON fields (e.g. cs10, cs11, additionalReqHeaders) by disabling RFC 4180 CSV boundary handling | | |
| | 3.1.1 | 09-04-2026 | Fixed **CCF connector** log ingestion for JSON fields containing quotation characters | |
…g drops (v3.1.1) The CCF CSV parser defaults to RFC 4180 mode (CsvHelper.CsvMode.RFC4180) where double-quote characters are treated as field enclosures. Imperva WAF CEF logs contain embedded JSON in fields like cs10, cs11, additionalReqHeaders, and additionalResHeaders which include quotation marks as regular data content. This caused the parser to silently drop log lines containing these characters before they reached the DCR. Fix: Added csvEscapeMode: NoEscape to the dataFormat configuration in both the PollingConfig and mainTemplate. This sets CsvHelper.CsvMode.NoEscape which disables all quote/escape character interpretation. CEF format uses backslash escaping, not RFC 4180 quote enclosure, making NoEscape the correct parser behavior for this data source. Validated against customer sample data (5 log lines, 4 previously dropped). No regression to existing ingestion of non-quoted log lines. Bumped solution version to 3.1.1 and created release package.
… fix/imperva-ccf-hasCsvBoundary-v3.1.1
v-krishnachi
force-pushed
the
fix/imperva-ccf-hasCsvBoundary-v3.1.1
branch
from
fd2b070 to
8538649
Compare
Contributor
Author
|
There is new PR given by Jason #14044 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CCF CSV parser defaults to RFC 4180 behavior where double-quote characters are treated as field enclosures.
Imperva WAF CEF logs contain embedded JSON in below fields which include quotation marks as regular data content.
1. cs10
2. cs11
3. additionalReqHeaders
4. additionalResHeaders
This caused the parser to silently drop log lines containing these characters before they reached the DCR.
Fix: Added HasCsvBoundary: false to the dataFormat configuration in both the PollingConfig and mainTemplate to disable RFC 4180 quote-boundary handling.
Validated against customer sample data (5 log lines, 4 previously dropped). No regression to existing ingestion of non-quoted log lines.
Bumped solution version to 3.1.1 and created release package.
Required items, please complete
Change(s):
Version updated: