Remove config.py in favor of diskcache-based JSON loading

[frack113]

Summary

This PR replaces the monolithic ConfigHQ singleton (which loaded all SigmaHQ data into memory at import time) with per-domain data modules backed by diskcache for lazy, cached loading. It also splits the overloaded field.py into focused modules and refactors the status validator to properly handle regression-test rules.

Closes #72.
Closes #74.

Motivation

The old ConfigHQ class in config.py:

• Loaded all taxonomy, filename patterns, Windows event IDs, and provider names into memory on every import
• Used a hardcoded sigmahq_data.py generated by a script (update_ref.py)
• Required manual regeneration when JSON data changed
• Had no caching — every Python process re-fetched remote data

The new architecture:

• Four standalone modules under sigma/validators/sigmahq/data/, one per data domain
• Uses diskcache to cache remote JSON locally (TTL-managed)
• Lazy loading via __getattr__ — data is fetched only when first accessed
• Each module supports set_url() (for custom/local URLs) and set_cache_dir()
• Cleaner separation of concerns

Changes

Core refactoring

Change	Details
🔥 Remove config.py, sigmahq_data.py, update_ref.py	Old ConfigHQ singleton and its data generator script deleted
✨ Add diskcache data modules	data_filename.py, data_taxonomy.py, data_windows_eventid.py, data_windows_provider.py — each lazy-loads from a JSON URL with local caching
♻️ Extract SigmahqDataLoader base class	New base.py with SigmahqDataLoader(ABC) — shared cache management, JSON fetching, URL/config handling, and __getattr__ delegation. Each data module becomes a thin subclass with only _parse()
♻️ make_module_api() helper	Factors globals().update(...) boilerplate into a single function in base.py — each module calls it in one line
♻️ Split field.py → fieldname.py + modifier.py	fieldname.py: field name/presence validators. modifier.py: modifier-specific validators (duplicate value, invalid all-modifier)
♻️ Update validators	detection.py, filename.py, logsource.py now import from data_* modules instead of ConfigHQ
♻️ Clean unused imports	tags.py, title.py: removed dead ConfigHQ imports
♻️ status.py: SigmahqStatusToHighValidator	Replaced single min_days with min_days_for_nolog_rule (60) + min_days_for_log_rule (0). Rules with regression_tests_path now use the lower log threshold. Early-return logic simplified. Fixes #72.

JSON schema & tools

Change	Details
🔧 Split windows_validator	Old single schema/data file split into windows_eventid (event ID categories) and windows_provider (provider names)
🔧 New schemas	schema_sigmahq_windows_eventid.json, schema_sigmahq_windows_provider.json

CI & docs

Change	Details
👷 Update schema.yml	Validates the two new JSON files against their schemas
📝 Add CHANGELOG.md, CONTRIBUTING.md	Basic project docs
📝 Update README.md	Reflects new data loading approach
🔖 Bump version to 0.21.0	Minor bump for significant refactoring

Bug fixes (from code review)

Change	Details
🐛 Type annotations	Fixed Dict[str, str] → Dict[str, Any] in eventid/provider modules; fixed Dict[str, List[str]] → Dict[str, str] in filename module
🐛 Key safety	Replaced bare json_data["key"] with .get(key, default) to prevent KeyError on malformed JSON
🐛 Cache lifecycle	set_url() now resets _cache = None consistently across all 4 modules
🐛 HTTP errors	Added http.client.HTTPException to except tuples to catch IncompleteRead and similar stream errors
🐛 Nested field safety	data_taxonomy.py uses info.get("field", {}).get("native", []) instead of bare indexing
🐛 JSON disk serializer	Added _JSONDisk custom serializer in base.py so non-root users can cache JSON (fixes pickling issues)
🐛 AttributeError module name	Fixed __name__ reference in base class to point to the correct subclass module

Test infrastructure

Change	Details
✅ conftest.py	New top-level conftest redirects data modules to local tools/*.json files during testing, avoiding 404 errors on CI (related to #74 — demonstrates offline/local usage)
♻️ Test reorganization	Tests moved from tests/sigmahq/field/ to tests/sigmahq/fieldname/ and tests/sigmahq/modifier/; fixed imports; added parametrized tests for status validator

[thomaspatzke]

Most of the code repeats in the other data modules. This redundancy can cause inconsistencies. Suggestion: transform into classes with a base class containing shared functions and implement only differences in specific classes.

[frack113]

Thanks for the review. It's done .