Correct correlation-rule Event Query hydration guidance (#28)

The skills stated that correlation-rule detections cannot be hydrated by an Event Query and must use a Get Detection Details action. Community evidence from working production workflows shows this is wrong: correlation-rule detections hydrate with the same Ngsiem.alert.id = ?DetectionID query as first- and third-party detections. The catch is that the query returns multiple records (the alert record plus the underlying events), so an unfiltered results[0] is non-deterministic. Filtering with | #event.kind = "event" or | Ngsiem.event.product = CrowdStrike, or projecting named columns with table([...]), narrows it to one predictable row.

Updates the two trigger-types.md copies, event-query-vs-api.md, event-queries.md, and the ngsiem-detection-response use case to say hydration works for all detection types with the filtering caveat, keeping Get Detection Details as a valid fallback when Event Query results are unreliable.

The validate.py Ngsiem.detection.id join guard is unchanged in behavior (that field is the wrong join key for any detection type), but its error message no longer tells authors correlation-rule detections need Get Detection Details; it now points them to Ngsiem.alert.id with the multi-record filtering note. 541 tests pass; pylint and markdownlint clean.

Author: mraible

skills/authoring/references/event-query-vs-api.md

@@ -81,16 +81,17 @@ silently return zero. The split is about whether you already hold the object:
     Ngsiem.detection.id = ?detectID
     ```
 
-  - **Correlation-rule detections** → **Get Detection Details** action, NOT an
-    Event Query. The rule trigger exposes only the composite `DetectionID` and an
-    opaque `ID`; neither matches the `Ngsiem.detection.id` value that Advanced
-    event search filters on, and that field is not in the trigger schema. The
-    field that used to bridge it (`Ngsiem.original_indicator.id`) is gone from the
-    rule-trigger events, so a correlation-rule detection cannot be hydrated via
-    Event Query today. Add a **Get Detection Details** gather step (or a
-    CrowdStrike HTTP Request to `/alerts/entities/alerts/v2` passing the composite
-    `DetectionID` as `composite_id`) to fetch the full detection, then read fields
-    from its response.
+  - **Correlation-rule detections** → hydrate with the same
+    `Ngsiem.alert.id = ?detectID` query. It works, but returns **multiple records**
+    (the alert record plus the underlying events), so `results[0]` is
+    non-deterministic across runs. Filter to a single predictable row with
+    `| #event.kind = "event"` or `| Ngsiem.event.product = CrowdStrike`, or project
+    named columns with `table([field1, field2, ...])`. Restrict the action's output
+    schema to only the fields you read, so runs that omit some fields don't fail
+    schema validation. If Event Query results are unreliable for your detection
+    type, a **Get Detection Details** action (or a CrowdStrike HTTP Request to
+    `/alerts/entities/alerts/v2` passing the composite `DetectionID` as
+    `composite_id`) is a valid alternative.
 
   **Treat detection IDs as opaque.** Per the detections team, `composite_id` (what
   the trigger hands you) is the primary key used to retrieve a detection, but its

skills/authoring/references/trigger-types.md

@@ -214,13 +214,17 @@ list), hydrate with an Event Query and read from its `results` array:
 through an inline Python extractor** (`cs.json.decode(data['<Python>.output_stdout'])`)
 — that form does not resolve at release. Read `results[0].<Field>` directly.
 
-**Event Query hydration only works for first-party and third-party detections**
-(match the composite `DetectionID` against `Ngsiem.alert.id`, not
-`Ngsiem.detection.id`). **Correlation-rule detections cannot be hydrated by an
-Event Query** — their `Ngsiem.detection.id` is not exposed on the trigger and no
-Event Query field matches it. For those, add a **Get Detection Details** action
-(or an HTTP Request to `/alerts/entities/alerts/v2` passing the composite
-`DetectionID` as `composite_id`). See `references/event-query-vs-api.md`.
+**Event Query hydration works for all NG-SIEM detection types** — match the
+composite `DetectionID` against `Ngsiem.alert.id` (not `Ngsiem.detection.id`).
+For **correlation-rule detections** the same `Ngsiem.alert.id = ?DetectionID`
+query works, but it returns **multiple records** (the alert record plus the
+underlying events), so `results[0]` is non-deterministic without a filter. Narrow
+it to one predictable row with `| #event.kind = "event"` or
+`| Ngsiem.event.product = CrowdStrike`, or project named columns with
+`table([field1, field2, ...])`. If Event Query results are unreliable for a given
+detection type, a **Get Detection Details** action (or an HTTP Request to
+`/alerts/entities/alerts/v2` passing the composite `DetectionID` as `composite_id`)
+is a valid alternative. See `references/event-query-vs-api.md`.
 
 **Not every `Trigger.Detection.*` field resolves on the NG-SIEM trigger.**
 `${Trigger.Detection.Product}` and `${Trigger.Detection.Description}` are available

skills/authoring/scripts/validate.py

@@ -421,12 +421,11 @@ def _check_detection_hydration_join(label, config, issues):
         issues.append(
             f"ERROR: Event Query action '{label}' hydrates a detection by "
             f"matching 'Ngsiem.detection.id = ?...'. The trigger never provides "
-            f"that field. For first-party and third-party detections, the "
-            f"trigger's composite DetectionID is stored in 'Ngsiem.alert.id' — "
-            f"change the join field to 'Ngsiem.alert.id'. For correlation-rule "
-            f"detections, neither Event Query field matches; use a 'Get Detection "
-            f"Details' action (or an HTTP Request to /alerts/entities/alerts/v2 "
-            f"passing the composite DetectionID as composite_id) instead. As "
+            f"that field. The trigger's composite DetectionID is stored in "
+            f"'Ngsiem.alert.id' for all detection types — change the join field to "
+            f"'Ngsiem.alert.id'. For correlation-rule detections that query returns "
+            f"multiple records (the alert plus underlying events), so filter it "
+            f"(e.g. '| #event.kind = \"event\"') to a single predictable row. As "
             f"written this query returns zero rows at runtime and the workflow "
             f"enriches nothing, despite releasing cleanly. See "
             f"references/event-query-vs-api.md."

skills/workflows/references/trigger-types.md

@@ -123,14 +123,16 @@ rejected at release as "unknown variable" for EPP triggers. Enrich indicators
 straight from the trigger payload (e.g. send `Process.SHA256` to VirusTotal);
 you do not need an Event Query to hydrate the detection.
 
-**If you must hydrate a detection** (indicators not on the trigger), the path
-depends on detection type. First-party/third-party detections can be pulled with
-an Event Query matching the composite `DetectionID` against `Ngsiem.alert.id` (not
-`Ngsiem.detection.id`). **Correlation-rule detections cannot be hydrated by an
-Event Query** — their `Ngsiem.detection.id` is not exposed on the trigger; use a
-**Get Detection Details** action (or an HTTP Request to
+**If you must hydrate a detection** (indicators not on the trigger), match the
+composite `DetectionID` against `Ngsiem.alert.id` (not `Ngsiem.detection.id`) in
+an Event Query. This works for all NG-SIEM detection types. For **correlation-rule
+detections** the same query returns **multiple records** (the alert plus the
+underlying events), so filter it to one predictable row — `| #event.kind = "event"`
+or `| Ngsiem.event.product = CrowdStrike` — or project columns with `table([...])`.
+A **Get Detection Details** action (or an HTTP Request to
 `/alerts/entities/alerts/v2` passing the composite `DetectionID` as `composite_id`)
-instead. See `../../authoring/references/event-query-vs-api.md`.
+is a valid alternative if Event Query results are unreliable. See
+`../../authoring/references/event-query-vs-api.md`.
 
 **Severity is an integer (1-5) at `Trigger.Detection.Severity`, not a string.** Use numeric comparison in CEL conditions:
 

use-cases/event-queries.md

@@ -12,7 +12,8 @@ User wants a workflow to query the NG-SIEM event store — ingested logs, custom
 LogScale search results, or enriching a detection the workflow already holds (e.g.
 `Ngsiem.alert.id = ?detectID` — for first-party/third-party detections the trigger's composite
 `DetectionID` is stored as `Ngsiem.alert.id`, not `Ngsiem.detection.id`; correlation-rule
-detections can't be hydrated this way, use a Get Detection Details action) — without defining a
+detections can be hydrated the same way, but the query returns multiple records — filter it, see
+below) — without defining a
 schema up front. The Event Query action
 runs a CQL/FQL query inline and
 returns matching events, which subsequent actions process and branch on. This is the
@@ -109,6 +110,9 @@ code-driven querying belongs in a function.
 > you already hold* — when the workflow was triggered on a **first-party or third-party** detection
 > and has its ID, an Event Query like `Ngsiem.alert.id = ?detectID` to pull more fields is the right
 > tool (match the composite `DetectionID` against `Ngsiem.alert.id`, not `Ngsiem.detection.id`).
-> **Correlation-rule detections are the exception:** their `Ngsiem.detection.id` is not in the
-> trigger and can't be reached by an Event Query — use a **Get Detection Details** action instead.
-> See [event-query-vs-api.md](../skills/authoring/references/event-query-vs-api.md).
+> **Correlation-rule detections can also be hydrated** with `Ngsiem.alert.id = ?detectID`, but the
+> query returns multiple records (the alert plus the underlying events), so `results[0]` is
+> non-deterministic without a filter. Narrow it with `| #event.kind = "event"` or
+> `| Ngsiem.event.product = CrowdStrike`, or project named columns with `table([...])`. A **Get
+> Detection Details** action remains a valid alternative if Event Query results are unreliable for
+> your detection type. See [event-query-vs-api.md](../skills/authoring/references/event-query-vs-api.md).

use-cases/ngsiem-detection-response.md

@@ -76,11 +76,13 @@ where each is covered in depth and verified live.
 
 ## Gotchas
 
-- **Hydration join key is detection-type-dependent.** `Ngsiem.alert.id` works for
-  first/third-party detections (and for querying a correlation detection's
-  underlying connector events); a correlation *detection record* with no queryable
-  underlying events needs a Get Detection Details action instead. This is the most
-  common silent-empty failure - see `references/event-query-vs-api.md`.
+- **Hydration join key is `Ngsiem.alert.id`, not `Ngsiem.detection.id`.** Match the
+  composite `DetectionID` against `Ngsiem.alert.id` for all detection types. A
+  correlation-rule detection hydrates the same way, but the query returns multiple
+  records (the alert plus its underlying events), so filter to one predictable row
+  (`| #event.kind = "event"` or `| Ngsiem.event.product = CrowdStrike`) — an
+  unfiltered `results[0]` is the most common silent-empty/wrong-row failure. A Get
+  Detection Details action is a valid fallback. See `references/event-query-vs-api.md`.
 - **Charlotte AI needs credits**, and the org must opt in. A workflow that invokes
   it will not produce a summary in a tenant without them.
 - **Business-hours logic:** the guide implements the time gate as hardcoded