Skip to content

Remove config to use cache - #69

Merged
frack113 merged 12 commits into
SigmaHQ:v0.30.0from
frack113:Refractor
Apr 9, 2026
Merged

Remove config to use cache#69
frack113 merged 12 commits into
SigmaHQ:v0.30.0from
frack113:Refractor

Conversation

@frack113

@frack113 frack113 commented Dec 7, 2025

Copy link
Copy Markdown
Member

Breack logic to use diskcache in v0.30.0

  • Use diskcache create:
    • data_filename
    • data_taxonomy
    • data_windows_eventid
    • data_windows_provider
  • Split sigmahq_windows_validator.json into
    • sigmahq_windows_eventid.json
    • sigmahq_windows_provider.json
  • Update json schema
  • rename field.py to fieldname.py
  • Remove dead code
    • config.py
    • update_ref.py
    • test_config.py
    • sigmahq_data.py

@frack113
frack113 marked this pull request as draft December 7, 2025 13:01
@frack113
frack113 marked this pull request as ready for review December 7, 2025 13:30
@frack113
frack113 changed the base branch from main to v0.30.0 December 10, 2025 05:30
@frack113 frack113 moved this to In review in Version 0.30.0 Dec 12, 2025
@nasbench
nasbench requested a review from Copilot December 24, 2025 15:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the SigmaHQ validator to use diskcache for dynamic data loading instead of static generated Python files. The changes improve the architecture by:

  • Implementing disk caching for remote JSON data from the SigmaHQ repository
  • Splitting Windows validator data into separate eventid and provider JSON files
  • Removing dead code including the config system and data generation scripts
  • Updating validators to use the new data loading modules

Reviewed changes

Copilot reviewed 30 out of 31 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
sigma/validators/sigmahq/data/data_taxonomy.py New module for loading and caching taxonomy data with disk cache
sigma/validators/sigmahq/data/data_filename.py New module for loading and caching filename pattern data
sigma/validators/sigmahq/data/data_windows_eventid.py New module for Windows EventID validation data
sigma/validators/sigmahq/data/data_windows_provider.py New module for Windows provider validation data
sigma/validators/sigmahq/fieldname.py Updated to use new data_taxonomy module instead of config
sigma/validators/sigmahq/logsource.py Updated to use new data_taxonomy module
sigma/validators/sigmahq/filename.py Updated to use new data_filename module
sigma/validators/sigmahq/detection.py Updated to use new Windows data modules
sigma/validators/sigmahq/modifier.py New module containing field duplicate and all modifier validators
tools/sigmahq_windows_eventid.json New JSON file separating EventID validation data
tools/sigmahq_windows_provider.json Updated JSON file with only provider data
json-schema/schema_sigmahq_windows_eventid.json New JSON schema for EventID validation
json-schema/schema_sigmahq_windows_provider.json Updated schema removing eventid field
tests/* Updated test imports from field to fieldname/modifier modules

Comment thread sigma/validators/sigmahq/data/data_filename.py Outdated
Comment thread sigma/validators/sigmahq/data/data_taxonomy.py Outdated
Comment thread sigma/validators/sigmahq/data/data_taxonomy.py Outdated
Comment thread sigma/validators/sigmahq/data/data_filename.py Outdated
Comment thread sigma/validators/sigmahq/data/data_windows_eventid.py Outdated
Comment thread sigma/validators/sigmahq/data/data_windows_eventid.py Outdated
Comment thread sigma/validators/sigmahq/data/data_windows_provider.py Outdated
Comment thread sigma/validators/sigmahq/data/data_windows_provider.py Outdated
Comment thread sigma/validators/sigmahq/modifier.py Outdated
Comment thread sigma/validators/sigmahq/logsource.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@frack113

Copy link
Copy Markdown
Member Author

Test will not pass as the json are not in the main branch by now 😞
Wait for a new repo to put them.

@frack113
frack113 merged commit fdd8212 into SigmaHQ:v0.30.0 Apr 9, 2026
1 of 21 checks passed
@frack113
frack113 deleted the Refractor branch April 9, 2026 16:36
@github-project-automation github-project-automation Bot moved this from In review to Done in Version 0.30.0 Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants