Skip to content

TECH_DEBT: Add SKIP/SUPPRESS strategies to validation categorization - #1684

Draft
c-schuler wants to merge 21 commits into
devfrom
validation-config
Draft

TECH_DEBT: Add SKIP/SUPPRESS strategies to validation categorization#1684
c-schuler wants to merge 21 commits into
devfrom
validation-config

Conversation

@c-schuler

@c-schuler c-schuler commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

🛠️ Description of Changes

Promotes categorization rules from post-validation labels to advisor-driven strategies that drop noise before it reaches OperationOutcome. Implements four phases of the roadmap in Java/validation/VALIDATION-CATEGORIES-DESIGN.md: the SKIP and SUPPRESS strategies, two narrowing axes per strategy, and seven rule migrations.

  • SKIP strategy at HAPI's policyForCodedContent hook - short-circuits terminology validation before it runs. Two narrowing axes:
    • scope.codeSystems - fires only when the code system URL matches
    • scope.excludeActions - fires every call but removes only the named CodedContentValidationActions instead of every action
    • SUPPRESS strategy at HAPI's isSuppressMessageId hook - drops a message after the check ran but before it reaches OperationOutcome. Two axes:
      • suppressMessageIds - list of stable I18nConstants IDs to drop
      • suppressPathPatterns - regex narrowing on the FHIR path the message would fire on
    • Per-rule observability via a new link.validation.rule.outcome counter tagged with rule_id and outcome (one of skipped, suppressed, labeled). CategorizationService increments labeled for every post-validation match; the advisor increments skipped / suppressed on its respective hooks. Anyone with Grafana access can see, per rule, how often each lifecycle stage actually does work.
    • Schema infrastructure: new Category columns (strategy, scope, suppress_message_ids, suppress_path_patterns), JPA converters, CategorySnapshot mirroring, and two pairs of forward+undo migration scripts for SQL Server.
    • Design doc (Java/validation/VALIDATION-CATEGORIES-DESIGN.md) - phased roadmap, schema reference, migration log, related-code map. Documents not just what shipped but the design refinements that happened mid-phase (the excludeActions mechanism extension that emerged in Phase 2, the suppressPathPatterns extension in Phase 4, the Phase 3 deferral for lack of candidates).

Expected production impact

After deploy and advisor reload:

  • OperationOutcome payloads shrink on bundles that previously emitted the rule's messages - the messages are gone, not just tagged as acceptable
  • Result DB row count drops for the migrated rule IDs; downstream ValidationComplete signals are unaffected (the valid boolean computation is unchanged because the now-suppressed rules were all acceptable: true)
  • link.validation.rule.outcome Grafana panel shows the work distribution per rule - skipped and suppressed streams start firing for the migrated rules, labeled drops close to zero except on cold start

CPU impact on terminology validation specifically is modest because the team's existing remote-TS response caching already eliminates most network cost on the SKIP-targeted message families. The headline benefits are signal-to-noise — fewer messages downstream consumers have to filter - and the Grafana visibility for tuning which rules to migrate next.

Wire-format / API impact

None for external consumers. The ValidationComplete Kafka message shape is unchanged. The validation service's REST API is unchanged. Internal-only changes: ValidationService constructor gains a CategoryBackedPolicyAdvisor injection; CategorizationService gains a ValidationMetrics injection.

Database migrations

Two pairs of V/U scripts:

  • V20260616__Add_category_strategy_and_scope.sql - adds strategy (NOT NULL DEFAULT 'LABEL') and scope (nullable JSON) columns. Existing rows backfill to LABEL so the change is behaviour-preserving.
  • V20260616_2__Add_category_suppress_message_ids.sql - adds suppress_message_ids (nullable JSON)
  • V20260623__Add_category_suppress_path_patterns.sql - adds suppress_path_patterns (nullable JSON)

All migrations use defensive if not exists (select 1 from sys.columns ...) guards. Matching U*.sql undo scripts ship alongside.

In environments using spring.jpa.hibernate.ddl-auto: update (the docker profile), Hibernate handles the column adds automatically. In production environments where ddl-auto defaults to none, the V scripts are the only path. Open item: confirm who runs these scripts in prod - flagged in the design doc as an unresolved follow-up from Phase 1.

🧪 Testing Performed

98 validation tests, all green. Started at 50 before this work.

  • 11 DebugSectionsTest-equivalent unit tests for CategoryStrategy + 5 for CategoryScopeConverter + 5 for SuppressMessageIdsConverter - schema/serialization correctness
  • 36 CategoryBackedPolicyAdvisorTest cases covering both hooks across the load-time validation matrix (acceptable=false demote, invalid regex demote, partial vs total failure, narrow + broad composability)
  • Existing tests updated where signatures changed (notably CategorizationServiceTest's constructor mock)

🧑‍🔬 Unit Testing

  • I have written or updated unit tests to cover my changes
  • Coverage: 0.0%

📓 Documentation Updated

Java/validation/VALIDATION-CATEGORIES-DESIGN.md (517 lines) is the source of truth and contains schema reference, phased roadmap, migration log, related-code map. Anyone joining the work should read that first. Worth a few minutes from reviewers as well. It captures the design decisions made during this PR (including the ones I'd have made differently in hindsight, with the rationale preserved).

c-schuler added 17 commits June 16, 2026 10:04
Captures the design and phased rollout plan for promoting categories.json rules from pure post-validation labels to a tiered set of strategies that can also skip work at the HAPI extension-point level. Pure documentation — no code change.
Lays in the type and storage infrastructure for per-rule handling strategies (SKIP / SUPPRESS / LABEL) without yet wiring the new fields into the validation lifecycle.
Lands the policy-advisor mechanism that lets SKIP-strategy categories short-circuit HAPI's coded-content validation before the underlying terminology call is made.
@c-schuler c-schuler self-assigned this Jun 23, 2026
@c-schuler c-schuler added the enhancement New feature or request label Jun 23, 2026
@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 39718428-4943-4707-ac05-c6a85957ad0f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch validation-config

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@c-schuler
c-schuler marked this pull request as draft June 23, 2026 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

db-change enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants