Skip to content

Add CMS-HCC E2E risk-score validation test and CI (#1294)#1351

Open
gopidaxhealthcaresolutions wants to merge 2 commits into
tuva-health:mainfrom
gopidaxhealthcaresolutions:feat/1294-cms-hcc-e2e-validation
Open

Add CMS-HCC E2E risk-score validation test and CI (#1294)#1351
gopidaxhealthcaresolutions wants to merge 2 commits into
tuva-health:mainfrom
gopidaxhealthcaresolutions:feat/1294-cms-hcc-e2e-validation

Conversation

@gopidaxhealthcaresolutions

Copy link
Copy Markdown

Closes #1294

Adds a CMS-HCC end-to-end risk-score validation test plus a CI workflow that runs it whenever the HCC marts (or their coefficient seeds) change.

What it validates

Two golden-patient dbt unit tests on cms_hcc__patient_risk_scores_monthly_by_factor_type — the model where every numeric transformation in the mart happens. They pin known inputs to hand-computed expected scores covering the full arithmetic:

  • raw coefficient summation
  • the v24/v28 transition blend weighting
  • normalization (raw / normalization_factor)
  • the payment adjustment (× (1 − ma_coding_pattern_adjustment))
  • the blend pivot into separate v24_risk_score / v28_risk_score columns
  • member-month weighting

Patient A exercises a pre-transition payment year (V24 only, with the normalization factor and coding-pattern adjustment driven by the adjustment_rates seed). Patient B exercises the 2024 transition year (V24 67% / V28 33% blend).

Expected values are exact 3-decimal results chosen to avoid half-way rounding boundaries so the tests are stable across warehouses.

Methodology validated against the docs

The fixtures match the Tuva CMS-HCCs mart docs and risk-adjustment knowledge page:

  • Transition schedule — PY2024 = 67% V24 / 33% V28, PY2025 = 33% / 67%, PY2026+ = 100% V28.
  • Payment formula — (raw_risk_score / normalization_factor) × (1 − 0.059), i.e. the documented worked example (0.754 / 1.015) × (1 − 0.059) = 0.699. Patient B asserts this same arithmetic for the blended path.

CI

.github/workflows/cms_hcc_e2e_validation.yml:

  • Triggers on pull_request to main when models/data_marts/cms_hcc/** or seeds/value_sets/cms_hcc/** change (plus manual workflow_dispatch).
  • Runs the cms_hcc-tagged unit tests on a local DuckDB database — no warehouse credentials, so it also runs on fork PRs.

The existing cms_hcc__int_members unit test is tagged cms_hcc so it runs in the same workflow.

Testing

All three cms_hcc-tagged unit tests pass locally on DuckDB via the exact CI selector (dbt test --select "tag:cms_hcc,test_type:unit").

Notes for reviewers

  • The model currently hardcodes the 2024–2027 normalization factors and coding-pattern adjustment in CASE expressions (cms_hcc__patient_risk_scores_monthly_by_factor_type.sql, with a TODO: Uncomment when seed is updated). Patient B asserts against those hardcoded transition-year values; when the seed-driven refactor lands, those expectations may need a refresh.
  • A natural follow-up (per the issue discussion) is validating against the official CMS reference output once that reference data is available.

Adds an end-to-end validation of the CMS-HCC risk-score arithmetic plus a
CI workflow that runs it whenever the HCC marts or their coefficient seeds
change.

Validation (models/data_marts/cms_hcc/cms_hcc__patient_risk_scores_unit_tests.yml):
two golden-patient dbt unit tests on cms_hcc__patient_risk_scores_monthly_by_factor_type,
the model where all scoring math happens. They pin known inputs to
hand-computed expected scores covering:
  - raw coefficient summation
  - the v24/v28 transition blend weighting (CMS PY2024 67%/33% schedule)
  - normalization (seed-driven and transition-year factors)
  - the ma_coding_pattern_adjustment payment step
  - the blend pivot into separate v24/v28 columns
  - member-month weighting
Expected values are exact 3-decimal results chosen to avoid half-way
rounding boundaries so the tests are stable across warehouses.

CI (.github/workflows/cms_hcc_e2e_validation.yml): triggers on pull_request
when models/data_marts/cms_hcc/** or seeds/value_sets/cms_hcc/** change (plus
manual workflow_dispatch), and runs the cms_hcc-tagged unit tests on a local
DuckDB database. No warehouse credentials required, so it also runs on fork PRs.

The existing cms_hcc__int_members unit test is tagged cms_hcc so it runs in
the same workflow.

A future enhancement is validating against the official CMS reference output
(per the issue discussion) once that reference data is available.
@netlify

netlify Bot commented Jun 22, 2026

Copy link
Copy Markdown

Deploy Preview for thetuvaproject canceled.

Name Link
🔨 Latest commit b9f39f6
🔍 Latest deploy log https://app.netlify.com/projects/thetuvaproject/deploys/6a4725e024e4aa0008199468

@chase-jones
chase-jones self-requested a review July 3, 2026 13:00
@chase-jones chase-jones self-assigned this Jul 3, 2026
@chase-jones chase-jones added community Issues created by community members enhancement New feature or request labels Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community Issues created by community members enhancement New feature or request

Projects

Status: 👀 Ready for Review

Development

Successfully merging this pull request may close these issues.

Create CMS-HCC CI E2E validation test

2 participants