Align correlation-rule hydration with documented filter and clarify Event Query vs Get Detection Details (#29)

Two refinements to the correlation-rule guidance. First, the multi-record filter now matches the pattern in the official Fusion detection-email documentation: drop the correlation meta-event and keep the underlying events with `| xdr_type != correlation-rule-detection | report_name != *`, rather than the inferred `#event.kind`/`Ngsiem.event.product` filters. Second, the choice between Event Query and Get Detection Details is reframed as a data-granularity decision, not a reliability one: Event Query reaches the event-level detail (per-event source IP, country, and so on) that composed the detection, while Get Detection Details returns the detection object's summary fields. Use whichever matches the data you actually need.

Updated event-query-vs-api.md, both trigger-types.md copies, event-queries.md, ngsiem-detection-response.md, and the validate.py guard message. 541 tests pass; pylint and markdownlint clean.

Author: mraible

skills/authoring/references/event-query-vs-api.md

@@ -83,15 +83,24 @@ silently return zero. The split is about whether you already hold the object:
 
   - **Correlation-rule detections** → hydrate with the same
     `Ngsiem.alert.id = ?detectID` query. It works, but returns **multiple records**
-    (the alert record plus the underlying events), so `results[0]` is
-    non-deterministic across runs. Filter to a single predictable row with
-    `| #event.kind = "event"` or `| Ngsiem.event.product = CrowdStrike`, or project
-    named columns with `table([field1, field2, ...])`. Restrict the action's output
-    schema to only the fields you read, so runs that omit some fields don't fail
-    schema validation. If Event Query results are unreliable for your detection
-    type, a **Get Detection Details** action (or a CrowdStrike HTTP Request to
-    `/alerts/entities/alerts/v2` passing the composite `DetectionID` as
-    `composite_id`) is a valid alternative.
+    (the underlying events plus a correlation "meta-event" that only signals the
+    rule fired), so `results[0]` is non-deterministic across runs. Drop the
+    meta-event and keep the real events:
+
+    ```
+    Ngsiem.alert.id = ?detectID
+    | xdr_type != correlation-rule-detection
+    | report_name != *
+    ```
+
+    or project named columns with `table([field1, field2, ...])`. Restrict the
+    action's output schema to only the fields you read, so runs that omit some
+    fields don't fail schema validation. Event Query is how you reach the
+    **event-level detail** (per-event source IP, country, and so on) that made up
+    the detection. A **Get Detection Details** action returns the detection
+    *object* instead — reach for it when the object's summary fields are all you
+    need, or call `/alerts/entities/alerts/v2` with the composite `DetectionID` as
+    `composite_id` for that same object.
 
   **Treat detection IDs as opaque.** Per the detections team, `composite_id` (what
   the trigger hands you) is the primary key used to retrieve a detection, but its

skills/authoring/references/trigger-types.md

@@ -217,14 +217,16 @@ through an inline Python extractor** (`cs.json.decode(data['<Python>.output_stdo
 **Event Query hydration works for all NG-SIEM detection types** — match the
 composite `DetectionID` against `Ngsiem.alert.id` (not `Ngsiem.detection.id`).
 For **correlation-rule detections** the same `Ngsiem.alert.id = ?DetectionID`
-query works, but it returns **multiple records** (the alert record plus the
-underlying events), so `results[0]` is non-deterministic without a filter. Narrow
-it to one predictable row with `| #event.kind = "event"` or
-`| Ngsiem.event.product = CrowdStrike`, or project named columns with
-`table([field1, field2, ...])`. If Event Query results are unreliable for a given
-detection type, a **Get Detection Details** action (or an HTTP Request to
-`/alerts/entities/alerts/v2` passing the composite `DetectionID` as `composite_id`)
-is a valid alternative. See `references/event-query-vs-api.md`.
+query works, but it returns **multiple records** (the underlying events plus a
+correlation "meta-event" that only signals the rule fired), so `results[0]` is
+non-deterministic without a filter. Drop the meta-event and keep the real events
+with `| xdr_type != correlation-rule-detection | report_name != *`, or project
+named columns with `table([field1, field2, ...])`. Event Query is how you reach
+the event-level detail that composed the detection; a **Get Detection Details**
+action (or an HTTP Request to `/alerts/entities/alerts/v2` passing the composite
+`DetectionID` as `composite_id`) returns the detection object instead — use it when
+the object's summary fields are all you need. See
+`references/event-query-vs-api.md`.
 
 **Not every `Trigger.Detection.*` field resolves on the NG-SIEM trigger.**
 `${Trigger.Detection.Product}` and `${Trigger.Detection.Description}` are available

skills/authoring/scripts/validate.py

@@ -424,8 +424,9 @@ def _check_detection_hydration_join(label, config, issues):
             f"that field. The trigger's composite DetectionID is stored in "
             f"'Ngsiem.alert.id' for all detection types — change the join field to "
             f"'Ngsiem.alert.id'. For correlation-rule detections that query returns "
-            f"multiple records (the alert plus underlying events), so filter it "
-            f"(e.g. '| #event.kind = \"event\"') to a single predictable row. As "
+            f"multiple records (the underlying events plus a correlation meta-event), "
+            f"so filter it (e.g. '| xdr_type != correlation-rule-detection | "
+            f"report_name != *') to the real events. As "
             f"written this query returns zero rows at runtime and the workflow "
             f"enriches nothing, despite releasing cleanly. See "
             f"references/event-query-vs-api.md."

skills/workflows/references/trigger-types.md

@@ -126,13 +126,15 @@ you do not need an Event Query to hydrate the detection.
 **If you must hydrate a detection** (indicators not on the trigger), match the
 composite `DetectionID` against `Ngsiem.alert.id` (not `Ngsiem.detection.id`) in
 an Event Query. This works for all NG-SIEM detection types. For **correlation-rule
-detections** the same query returns **multiple records** (the alert plus the
-underlying events), so filter it to one predictable row — `| #event.kind = "event"`
-or `| Ngsiem.event.product = CrowdStrike` — or project columns with `table([...])`.
-A **Get Detection Details** action (or an HTTP Request to
+detections** the same query returns **multiple records** (the underlying events
+plus a correlation "meta-event" that only signals the rule fired), so drop the
+meta-event and keep the real events —
+`| xdr_type != correlation-rule-detection | report_name != *` — or project columns
+with `table([...])`. Event Query is how you reach the event-level detail that made
+up the detection; a **Get Detection Details** action (or an HTTP Request to
 `/alerts/entities/alerts/v2` passing the composite `DetectionID` as `composite_id`)
-is a valid alternative if Event Query results are unreliable. See
-`../../authoring/references/event-query-vs-api.md`.
+returns the detection object instead — use it when the object's summary fields are
+all you need. See `../../authoring/references/event-query-vs-api.md`.
 
 **Severity is an integer (1-5) at `Trigger.Detection.Severity`, not a string.** Use numeric comparison in CEL conditions:
 

use-cases/event-queries.md

@@ -111,8 +111,10 @@ code-driven querying belongs in a function.
 > and has its ID, an Event Query like `Ngsiem.alert.id = ?detectID` to pull more fields is the right
 > tool (match the composite `DetectionID` against `Ngsiem.alert.id`, not `Ngsiem.detection.id`).
 > **Correlation-rule detections can also be hydrated** with `Ngsiem.alert.id = ?detectID`, but the
-> query returns multiple records (the alert plus the underlying events), so `results[0]` is
-> non-deterministic without a filter. Narrow it with `| #event.kind = "event"` or
-> `| Ngsiem.event.product = CrowdStrike`, or project named columns with `table([...])`. A **Get
-> Detection Details** action remains a valid alternative if Event Query results are unreliable for
-> your detection type. See [event-query-vs-api.md](../skills/authoring/references/event-query-vs-api.md).
+> query returns multiple records (the underlying events plus a correlation "meta-event" that only
+> signals the rule fired), so `results[0]` is non-deterministic without a filter. Drop the meta-event
+> and keep the real events with `| xdr_type != correlation-rule-detection | report_name != *`, or
+> project named columns with `table([...])`. Event Query is how you reach the event-level detail that
+> composed the detection; a **Get Detection Details** action returns the detection object instead —
+> use it when the object's summary fields are all you need. See
+> [event-query-vs-api.md](../skills/authoring/references/event-query-vs-api.md).

use-cases/ngsiem-detection-response.md

@@ -79,10 +79,12 @@ where each is covered in depth and verified live.
 - **Hydration join key is `Ngsiem.alert.id`, not `Ngsiem.detection.id`.** Match the
   composite `DetectionID` against `Ngsiem.alert.id` for all detection types. A
   correlation-rule detection hydrates the same way, but the query returns multiple
-  records (the alert plus its underlying events), so filter to one predictable row
-  (`| #event.kind = "event"` or `| Ngsiem.event.product = CrowdStrike`) — an
-  unfiltered `results[0]` is the most common silent-empty/wrong-row failure. A Get
-  Detection Details action is a valid fallback. See `references/event-query-vs-api.md`.
+  records (the underlying events plus a correlation "meta-event" that only signals
+  the rule fired), so drop the meta-event and keep the real events
+  (`| xdr_type != correlation-rule-detection | report_name != *`) — an unfiltered
+  `results[0]` is the most common silent-empty/wrong-row failure. Event Query is how
+  you reach the event-level detail; a Get Detection Details action returns the
+  detection object instead. See `references/event-query-vs-api.md`.
 - **Charlotte AI needs credits**, and the org must opt in. A workflow that invokes
   it will not produce a summary in a tenant without them.
 - **Business-hours logic:** the guide implements the time gate as hardcoded