Skip to content

Add rex extraction for scalar |re modifiers with named capture groups - #71

Open
SkxOverKill wants to merge 1 commit into
SigmaHQ:mainfrom
SkxOverKill:feat/rex-named-capture-groups
Open

Add rex extraction for scalar |re modifiers with named capture groups#71
SkxOverKill wants to merge 1 commit into
SigmaHQ:mainfrom
SkxOverKill:feat/rex-named-capture-groups

Conversation

@SkxOverKill

@SkxOverKill SkxOverKill commented Aug 12, 2026

Copy link
Copy Markdown

Closes #70

Summary

Extends the regex conversion for scalar |re modifiers (string values) so that regular expressions containing a named capture group ((?P<name>...)) are additionally converted to a Splunk rex extraction command, while preserving the existing matching semantics of the regex command.

Previously, scalar |re values were always emitted solely as a regex matching command, which cannot extract captured groups into fields. This change appends a rex field=<field> "(?<name>...)" command when a Python-style named capture group is present, converting the group to Splunk's PCRE syntax ((?<name>...)) for consistency with the group names the backend already generates (e.g. in the OR path).

Before

| regex User="(?P<user>\S+)"

After

| regex User="(?P<user>\S+)"
| rex field=User "(?<user>\S+)"

Design notes

  • The regex matching command is retained alongside rex. Splunk's rex command only extracts fields and does not filter events; emitting rex on its own would silently broaden the result set and regress detection quality. Keeping both commands preserves the exact match semantics while making the captured fields available downstream.
  • Group syntax is converted from Python ((?P<name>...)) to PCRE ((?<name>...)) so the extracted field names align with the conventions used by SplunkDeferredORRegularExpression.
  • Extraction is intentionally skipped for negated conditions (not <field>|re: ...), where a negated match has no usable capture, and for escaped group parentheses so literal matches are not rewritten.
  • List-valued |re modifiers (the OR path) are left unchanged.

Changes

  • sigma/backends/splunk/splunk.py
    • Add SplunkDeferredRexExpression, a deferred expression producing rex field=<field> "<pattern>".
    • Detect unescaped Python-style named capture groups in scalar |re values and append a rex extraction while keeping the existing regex matching command.
  • tests/test_backend_splunk.py
    • Add tests covering a named group alongside other fields, a standalone named group, multiple named groups, and the negated-condition case.

Testing

pytest tests129 passed (125 existing + 4 new), no warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Converting |re to Splunk rex command if type is String and a field ?<extracted_field> is defined

1 participant