Skip to content

Support enrichment fields in correlation aggregation queries - #69

Open
nephalemsec wants to merge 1 commit into
SigmaHQ:mainfrom
nephalemsec:feature-correlation-enrichment-fields
Open

Support enrichment fields in correlation aggregation queries#69
nephalemsec wants to merge 1 commit into
SigmaHQ:mainfrom
nephalemsec:feature-correlation-enrichment-fields

Conversation

@nephalemsec

Copy link
Copy Markdown

This adds support for the Sigma fields: attribute on correlation rules, so authors can carry additional context columns into the generated stats aggregation.

Today a value_count or event_count correlation renders only the aggregation metric and the group-by keys:

| stats dc(user) as value_count by _time device_id

Any fields listed on the correlation rule are dropped, so useful investigative context (a display name, source IP, and so on) has to be added by hand after conversion. With this change, fields declared on the correlation rule are emitted as values() clauses inside the same stats command:

| stats dc(user) as value_count values(device_name) as device_name values(src_ip) as src_ip by _time device_id

Implementation

  • Adds correlation_fields_expression, correlation_fields_field_expression, and correlation_fields_field_expression_joiner for the stats method, using the existing hook points in the core TextQueryBackend.
  • Adds a {fields} slot to the event_count and value_count aggregation templates.
  • Overrides convert_correlation_aggregation_fields_from_template so enrichment is drawn only from the correlation rule's own fields: list. Fields that appear in group-by are excluded, and the None/empty guards match the core implementation.

The Sigma correlation rules specification defines fields: on a correlation rule as the fields to output alongside the aggregation, separate from the fields: of the referenced rules. Scoping enrichment to the correlation rule's own list follows that definition.

Backwards compatibility

The change is additive. A correlation rule with no fields: produces byte-identical output to before, so existing rules and their expected queries are unaffected. The one deliberate design choice is that fields declared on referenced base rules are not pulled into the aggregation: only the correlation rule's own fields: drive enrichment. This keeps base rules (whose fields: normally feed a standalone | table) from silently changing correlation output.

Known limitation

Output columns are named after the field itself (values(fieldA) as fieldA), because the Sigma fields: list has no alias syntax. For fields whose names include characters that Splunk quotes (for example nested paths like target{}.displayName), the resulting column name carries that raw path. The query is valid and returns the expected values; only the column heading is verbose. Aliasing would need a mapping mechanism in the rule schema, which is out of scope here.

Tests

Adds coverage for value_count and event_count correlations with enrichment fields, a regression test asserting no change when fields: is absent, and a test confirming referenced base-rule fields do not leak into the aggregation. Existing tests are unchanged and all pass.

Support the Sigma fields: attribute on correlation rules so authors can carry
additional context columns into the generated stats aggregation. Fields declared
on the correlation rule are emitted as values() clauses alongside the aggregation
metric for the event_count and value_count stats methods.

Enrichment is scoped to the correlation rule's own fields: list; referenced
base-rule fields are not pulled in, so a correlation rule without its own fields:
produces byte-identical output to before. Existing tests are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant