Support enrichment fields in correlation aggregation queries - #69
Open
nephalemsec wants to merge 1 commit into
Open
Support enrichment fields in correlation aggregation queries#69nephalemsec wants to merge 1 commit into
nephalemsec wants to merge 1 commit into
Conversation
Support the Sigma fields: attribute on correlation rules so authors can carry additional context columns into the generated stats aggregation. Fields declared on the correlation rule are emitted as values() clauses alongside the aggregation metric for the event_count and value_count stats methods. Enrichment is scoped to the correlation rule's own fields: list; referenced base-rule fields are not pulled in, so a correlation rule without its own fields: produces byte-identical output to before. Existing tests are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds support for the Sigma
fields:attribute on correlation rules, so authors can carry additional context columns into the generatedstatsaggregation.Today a
value_countorevent_countcorrelation renders only the aggregation metric and the group-by keys:Any fields listed on the correlation rule are dropped, so useful investigative context (a display name, source IP, and so on) has to be added by hand after conversion. With this change, fields declared on the correlation rule are emitted as
values()clauses inside the samestatscommand:Implementation
correlation_fields_expression,correlation_fields_field_expression, andcorrelation_fields_field_expression_joinerfor thestatsmethod, using the existing hook points in the coreTextQueryBackend.{fields}slot to theevent_countandvalue_countaggregation templates.convert_correlation_aggregation_fields_from_templateso enrichment is drawn only from the correlation rule's ownfields:list. Fields that appear ingroup-byare excluded, and the None/empty guards match the core implementation.The Sigma correlation rules specification defines
fields:on a correlation rule as the fields to output alongside the aggregation, separate from thefields:of the referenced rules. Scoping enrichment to the correlation rule's own list follows that definition.Backwards compatibility
The change is additive. A correlation rule with no
fields:produces byte-identical output to before, so existing rules and their expected queries are unaffected. The one deliberate design choice is that fields declared on referenced base rules are not pulled into the aggregation: only the correlation rule's ownfields:drive enrichment. This keeps base rules (whosefields:normally feed a standalone| table) from silently changing correlation output.Known limitation
Output columns are named after the field itself (
values(fieldA) as fieldA), because the Sigmafields:list has no alias syntax. For fields whose names include characters that Splunk quotes (for example nested paths liketarget{}.displayName), the resulting column name carries that raw path. The query is valid and returns the expected values; only the column heading is verbose. Aliasing would need a mapping mechanism in the rule schema, which is out of scope here.Tests
Adds coverage for
value_countandevent_countcorrelations with enrichment fields, a regression test asserting no change whenfields:is absent, and a test confirming referenced base-rule fields do not leak into the aggregation. Existing tests are unchanged and all pass.