Fix broken tstats query when using regex modifier

[Copilot]

Deferred regex expressions (| rex, | eval, | regex) were being placed inside the tstats WHERE clause, producing invalid SPL:

| tstats ... where  | rex field=Processes.process "(?<processMatch>...)" | eval ... | search ...

Changes

• sigma/backends/splunk/splunk.py: finalize_query_data_model() now parses the query string to separate deferred expressions from the WHERE clause. OR regex prefixes (split at \n| search marker) and simple regex suffixes (split at \n| ) are extracted and placed after the tstats by clause.

• tests/test_backend_splunk.py: Added tests for simple regex AND and OR regex with contains in data_model format.

Result

| tstats ... where <conditions> by <fields>
| rex field=Processes.process "(?<processMatch>...)"
| eval processCondition=if(isnotnull(processMatch), "true", "false")
| search <conditions>
| `drop_dm_object_name(Processes)`
| convert ...

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[datamodel] broken tstats query when using regex</issue_title>
<issue_description>### Description

When converting rules with |re operator, the converted tstats query is invalid (WHERE expression issue).

For instance with this rule:

title: Suspicious LNK Command-Line Padding with Whitespace Characters
id: dd8756e7-a3a0-4768-b47e-8f545d1a751c
status: experimental
description: |
    Detects exploitation of LNK file command-line length discrepancy, where attackers hide malicious commands beyond the 260-character UI limit while the actual command-line argument field supports 4096 characters using whitespace padding (e.g., 0x20, 0x09-0x0D).
    Adversaries insert non-printable whitespace characters (e.g., Line Feed \x0A, Carriage Return \x0D) to pad the visible section of the LNK file, pushing malicious commands past the UI-visible boundary.
    The hidden payload, executed at runtime but invisible in Windows Explorer properties, enables stealthy execution and evasion—commonly used for social engineering attacks.
    This rule flags suspicious use of such padding observed in real-world attacks.
references:
    - https://syedhasan010.medium.com/forensics-analysis-of-an-lnk-file-da68a98b8415
    - https://thehackernews.com/2025/03/unpatched-windows-zero-day-flaw.html
    - https://www.trendmicro.com/en_us/research/25/c/windows-shortcut-zero-day-exploit.html
author: Swachchhanda Shrawan Poudel (Nextron Systems)
date: 2025-03-19
tags:
    - attack.initial-access
    - attack.execution
    - attack.t1204.002
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        - ParentImage|endswith: '\explorer.exe'
        - ParentCommandLine|contains: '.lnk'
    selection_cmd:
        - CommandLine|contains:
              - '                 '  # Padding of SPACE (0x20)
            # - '	'  # Horizontal Tab (0x9)
              - '\u0009'
              - '\u000A' # Line Feed
              - '\u0011'
              - '\u0012'
              - '\u0013'
              - '\u000B' # Vertical Tab
              - '\u000C'  # \x0C
              - '\u000D'  # \x0D
        - CommandLine|re: '\n\n\n\n\n\n' # In some cases \u000[ABCD] are represented as a newline in the eventlog
    condition: all of selection_*
falsepositives:
    - Unknown
level: high

We get: where | rex field=Processes.process "(?<processMatch>\\n\\n\\n\\n\\n\\n)" | eval processCondition=if(isnotnull(processMatch), "true", "false") | search

| tstats summariesonly=false allow_old_summaries=true fillnull_value="null" count min(_time) as firstTime max(_time) as lastTime from datamodel=Endpoint.Processes where  | rex field=Processes.process "(?<processMatch>\\n\\n\\n\\n\\n\\n)" | eval processCondition=if(isnotnull(processMatch), "true", "false") | search Processes.parent_process_path="*\\explorer.exe" OR Processes.parent_process="*.lnk*" Processes.process IN ("*                 *", "*\\u0009*", "*\\u000A*", "*\\u0011*", "*\\u0012*", "*\\u0013*", "*\\u000B*", "*\\u000C*", "*\\u000D*") OR processCondition="true" by Processes.process Processes.dest Processes.process_current_directory Processes.process_path Processes.process_integrity_level Processes.original_file_name Processes.parent_process Processes.parent_process_path Processes.parent_process_guid Processes.parent_process_id Processes.process_guid Processes.process_id Processes.user | `drop_dm_object_name(Processes)` | convert timeformat="%Y-%m-%dT%H:%M:%S" ctime(firstTime) | convert timeformat="%Y-%m-%dT%H:%M:%S" ctime(lastTime) 
```</issue_description>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>

• Fixes [datamodel] broken tstats query when using regex #60

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.