Add CMS-HCC E2E risk-score validation test and CI (#1294)#1351
Open
gopidaxhealthcaresolutions wants to merge 2 commits into
Open
Add CMS-HCC E2E risk-score validation test and CI (#1294)#1351gopidaxhealthcaresolutions wants to merge 2 commits into
gopidaxhealthcaresolutions wants to merge 2 commits into
Conversation
Adds an end-to-end validation of the CMS-HCC risk-score arithmetic plus a CI workflow that runs it whenever the HCC marts or their coefficient seeds change. Validation (models/data_marts/cms_hcc/cms_hcc__patient_risk_scores_unit_tests.yml): two golden-patient dbt unit tests on cms_hcc__patient_risk_scores_monthly_by_factor_type, the model where all scoring math happens. They pin known inputs to hand-computed expected scores covering: - raw coefficient summation - the v24/v28 transition blend weighting (CMS PY2024 67%/33% schedule) - normalization (seed-driven and transition-year factors) - the ma_coding_pattern_adjustment payment step - the blend pivot into separate v24/v28 columns - member-month weighting Expected values are exact 3-decimal results chosen to avoid half-way rounding boundaries so the tests are stable across warehouses. CI (.github/workflows/cms_hcc_e2e_validation.yml): triggers on pull_request when models/data_marts/cms_hcc/** or seeds/value_sets/cms_hcc/** change (plus manual workflow_dispatch), and runs the cms_hcc-tagged unit tests on a local DuckDB database. No warehouse credentials required, so it also runs on fork PRs. The existing cms_hcc__int_members unit test is tagged cms_hcc so it runs in the same workflow. A future enhancement is validating against the official CMS reference output (per the issue discussion) once that reference data is available.
✅ Deploy Preview for thetuvaproject canceled.
|
chase-jones
self-requested a review
July 3, 2026 13:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1294
Adds a CMS-HCC end-to-end risk-score validation test plus a CI workflow that runs it whenever the HCC marts (or their coefficient seeds) change.
What it validates
Two golden-patient dbt unit tests on
cms_hcc__patient_risk_scores_monthly_by_factor_type— the model where every numeric transformation in the mart happens. They pin known inputs to hand-computed expected scores covering the full arithmetic:raw / normalization_factor)× (1 − ma_coding_pattern_adjustment))v24_risk_score/v28_risk_scorecolumnsPatient A exercises a pre-transition payment year (V24 only, with the normalization factor and coding-pattern adjustment driven by the
adjustment_ratesseed). Patient B exercises the 2024 transition year (V24 67% / V28 33% blend).Expected values are exact 3-decimal results chosen to avoid half-way rounding boundaries so the tests are stable across warehouses.
Methodology validated against the docs
The fixtures match the Tuva CMS-HCCs mart docs and risk-adjustment knowledge page:
(raw_risk_score / normalization_factor) × (1 − 0.059), i.e. the documented worked example(0.754 / 1.015) × (1 − 0.059) = 0.699. Patient B asserts this same arithmetic for the blended path.CI
.github/workflows/cms_hcc_e2e_validation.yml:pull_requesttomainwhenmodels/data_marts/cms_hcc/**orseeds/value_sets/cms_hcc/**change (plus manualworkflow_dispatch).cms_hcc-tagged unit tests on a local DuckDB database — no warehouse credentials, so it also runs on fork PRs.The existing
cms_hcc__int_membersunit test is taggedcms_hccso it runs in the same workflow.Testing
All three
cms_hcc-tagged unit tests pass locally on DuckDB via the exact CI selector (dbt test --select "tag:cms_hcc,test_type:unit").Notes for reviewers
CASEexpressions (cms_hcc__patient_risk_scores_monthly_by_factor_type.sql, with aTODO: Uncomment when seed is updated). Patient B asserts against those hardcoded transition-year values; when the seed-driven refactor lands, those expectations may need a refresh.