v1.6.0: CADD v1.7 variant deleteriousness scores

PHRED-scaled deleteriousness enrichment from CADD v1.7. Two modes:
cache (pre-built SQLite, ~5GB, ~120M keys) and full (81GB tabix via
pysam, GRCh38 only). Enrichment-only annotator following the
gnomAD/AlphaMissense pattern. Opt-in — disabled by default.

Licensable-source gating: three-state permission model (ALLOW /
BLOCK_FINAL / BLOCK_PURCHASABLE) with per-source license assertions.
CADD is non-commercial by default but unlockable via commercial
license; SNPedia is permanently blocked in commercial mode.

Also: `config get` subcommand, strand normalization for array data,
CADD cache build script with int64 packing and indel pass.

1215 tests, 92%+ coverage.

Author: dial481

CHANGELOG.md

@@ -2,6 +2,33 @@
 
 All notable changes are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); the project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.6.0]
+
+### Added
+- **CADD v1.7 variant deleteriousness scores (ADR-0032).** PHRED-scaled
+  scores ranking how deleterious any single-nucleotide variant is, using
+  100+ annotation tracks. Enrichment-only annotator following the
+  gnomAD/AlphaMissense pattern. Two modes: cache (pre-built SQLite from
+  HuggingFace, ~5 GB, ~120M variant keys) and full (81 GB tabix file via pysam, GRCh38
+  only). CADD column appears in HTML, terminal, and JSON reports when
+  scores are present.
+- **Non-commercial source opt-in pattern.** CADD is the first source
+  with `commercial_ok=False`. Disabled by default (`sources.cadd =
+  false`). Users opt in via `allelix config set sources.cadd true` or
+  `allelix db update --cadd`. First download shows a license
+  confirmation prompt.
+- **Strand normalization for array data.** `resolve_strand()` maps
+  array-reported alleles to reference-forward orientation using gnomAD
+  ref/alt as ground truth. Palindromic SNPs (A/T, C/G) return None
+  rather than guessing.
+- **CADD cache build script.** `scripts/build_cadd_cache.py` filters
+  the full CADD SNV and indel files to positions present in gnomAD,
+  AlphaMissense, and ClinVar (GRCh38). Uses int64 packing for SNV
+  keys to fit the ~120M position set (117M SNV + 3M indel).
+- **`options.cadd_full` config key.** Enables full CADD mode (tabix
+  queries against the complete CADD file). Requires `pip install
+  allelix[cadd]` for pysam.
+
 ## [1.5.3]
 
 ### Fixed
@@ -1502,6 +1529,7 @@ All notable changes are documented here. Format follows [Keep a Changelog](https
 - GitHub Actions CI matrix on Python 3.11 and 3.12.
 
 
+[1.6.0]: https://github.com/dial481/allelix/compare/v1.5.3...v1.6.0
 [1.5.3]: https://github.com/dial481/allelix/compare/v1.5.2...v1.5.3
 [1.5.2]: https://github.com/dial481/allelix/compare/v1.5.1...v1.5.2
 [1.5.1]: https://github.com/dial481/allelix/compare/v1.5.0...v1.5.1

CONTRIBUTING.md

@@ -235,7 +235,7 @@ from __future__ import annotations
 import sqlite3
 from typing import TYPE_CHECKING, ClassVar
 
-from allelix.annotators.base import Annotator
+from allelix.annotators.base import Annotator, LicenseDescriptor
 from allelix.models import Annotation
 
 if TYPE_CHECKING:
@@ -249,6 +249,13 @@ class MyDBAnnotator(Annotator):
     display_name: ClassVar[str] = "MyDB"
     attribution: ClassVar[str] = "MyDB"
     requires_download: ClassVar[bool] = True
+    license: ClassVar[LicenseDescriptor] = LicenseDescriptor(
+        spdx="CC-BY-4.0",
+        license_url="https://example.com/mydb/license",
+        attribution_text="MyDB variant data.",
+        source_url="https://example.com/mydb",
+        commercial_ok=True,
+    )
 
     def __init__(self, data_dir: Path) -> None:
         super().__init__(data_dir)

README.md

@@ -3,13 +3,15 @@
 Open-source command-line toolkit for analyzing raw genotype files from consumer DNA testing services. Format-agnostic ingestion, database-agnostic annotation, offline-first.
 
 > **Status:** Production — six parser formats, four annotators (ClinVar +
-> PharmGKB + GWAS Catalog + SNPedia), two enrichment sources (gnomAD
-> population frequencies + AlphaMissense pathogenicity), dual-build
-> ClinVar caches (GRCh37 + GRCh38), HTML/JSON/terminal reports,
-> methylation + pharmacogenomics focused commands, report diffing,
-> persistent config with commercial-mode safety switch. Build
-> auto-detection from position data (ADR-0021). No regex on prose
-> anywhere in production. Release notes: [`CHANGELOG.md`](CHANGELOG.md).
+> PharmGKB + GWAS Catalog + SNPedia), three enrichment sources (gnomAD
+> population frequencies + AlphaMissense pathogenicity + CADD
+> deleteriousness), licensable-source gating for commercial users,
+> dual-build ClinVar caches (GRCh37 + GRCh38),
+> HTML/JSON/terminal reports, methylation + pharmacogenomics focused
+> commands, report diffing, persistent config with commercial-mode
+> safety switch. Build auto-detection from position data (ADR-0021).
+> No regex on prose anywhere in production. Release notes:
+> [`CHANGELOG.md`](CHANGELOG.md).
 
 ## Quickstart
 
@@ -31,6 +33,7 @@ allelix stats tests/fixtures/mock_myhappygenes.txt
 # Download reference databases. First run downloads all sources (~15GB
 # on disk with gnomAD + AlphaMissense). Use --no-gnomad / --no-alphamissense
 # to skip the large enrichment databases. Re-runs skip unchanged sources.
+# CADD is opt-in: allelix db update --cadd
 allelix db update
 allelix db status   # see what's cached
 
@@ -80,6 +83,7 @@ Adding a new format means adding one file to `allelix/parsers/` and registering
 | GWAS Catalog | ✓ | Public domain (EBI/NHGRI). Trait–SNP associations with p-values and effect sizes. Carrier rule (ADR-0007) requires the user to carry the risk allele. P-value magnitude scoring (ADR-0024) maps continuous p-values to the 0–10 scale; unknown-risk-allele entries fire on rsID match alone but are capped at 3.0. |
 | gnomAD | ✓ | ODbL v1.0. **Enrichment annotator** — adds population allele frequency context to existing annotations. Shows how common each variant is in the general population (~16M exome variants from 730K individuals). A pathogenic variant that 35% of people carry reads very differently from one seen in 0.001%. Pre-built cache downloaded via `db update` (~6GB on disk). Use `--no-gnomad` to skip. |
 | AlphaMissense | ✓ | CC BY 4.0. **Enrichment annotator** — adds DeepMind's protein-structure-based pathogenicity predictions to existing annotations. Scores 71M missense variants on a 0–1 scale: <0.34 = likely benign, >0.564 = likely pathogenic. Complements ClinVar's expert classifications with computational predictions — especially valuable for variants ClinVar hasn't reviewed yet. Pre-built cache downloaded via `db update` (~8GB on disk). Use `--no-alphamissense` to skip. |
+| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. Ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%. **Opt-in** — disabled by default (`sources.cadd = false`). Enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Pre-built cache (~5 GB on disk, ~120M variant keys). Full mode available via pysam for GRCh38 data (`options.cadd_full = true`). Cache mode covers the large majority of variants present in gnomAD, AlphaMissense, and ClinVar — nearly every position allelix can annotate from its other databases. For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), cache and full mode produce effectively identical results because chip probes overwhelmingly target known, cataloged variants. Full mode adds coverage for novel or private variants that appear only in whole-genome or whole-exome sequencing data and are not in any pre-computed database. If your input is a genotyping chip file, cache mode is all you need. |
 
 ### Known PharmGKB limitation: reference-genotype rows where ClinVar and CPIC both lack data
 
@@ -119,17 +123,24 @@ This is not a disclaimer afterthought. It is a design constraint that affects mo
 Allelix stores persistent configuration in `config.toml` (in the data directory, default `~/.local/share/allelix/`). A default config is created on first run.
 
 ```bash
-# View current config
+# View current config (annotated with license notes)
 allelix config show
 
+# Read a single key
+allelix config get sources.cadd
+allelix config get license.commercial
+
 # Disable a source permanently
 allelix config set sources.gnomad false
 
-# Enable commercial mode (auto-disables non-commercial sources like SNPedia)
+# Enable commercial mode (auto-disables non-commercial sources)
 allelix config set license.commercial true
+
+# Assert that you hold a commercial CADD license
+allelix config set license.cadd true
 ```
 
-CLI flags (`--no-gnomad`, `--no-alphamissense`, `--exclude-snpedia`) override the config for a single run. The config sets the baseline; flags override per-invocation.
+CLI flags (`--no-gnomad`, `--no-alphamissense`, `--exclude-snpedia`, `--cadd`) override the config for a single run. The config sets the baseline; flags override per-invocation.
 
 ### Database sizes and download times
 
@@ -142,6 +153,7 @@ Not all databases are equal in size. `allelix db update` downloads them all by d
 | GWAS Catalog | ~200MB | 1–2 min | Trait-SNP associations from genome-wide studies. |
 | gnomAD | ~6GB | 5–15 min | Population allele frequencies (how common is this variant?). |
 | AlphaMissense | ~8GB | 5–15 min | Missense pathogenicity predictions (how likely to break protein function?). |
+| CADD (opt-in) | ~5GB | 5–15 min | Variant deleteriousness scores (how damaging is this variant?). Enable with `--cadd`. |
 
 gnomAD and AlphaMissense are the largest but add the most interpretive context. gnomAD answers "is this variant rare or common?" — a pathogenic variant carried by 35% of the population reads very differently from one seen in 3 people. AlphaMissense answers "does this missense change likely damage the protein?" — especially valuable for the thousands of variants ClinVar hasn't reviewed yet.
 
@@ -160,8 +172,9 @@ Allelix source code is licensed under the **GNU Affero General Public License v3
 | SNPedia | snpedia.com | CC BY-NC-SA 3.0 US | Attribution required, **non-commercial only**. Use `--exclude-snpedia` to omit. |
 | gnomAD | gnomad.broadinstitute.org | ODbL v1.0 | Attribution required. Population allele frequencies for context; not a clinical annotator. Use `--no-gnomad` to omit. |
 | AlphaMissense | zenodo.org/records/10813168 | CC BY 4.0 | Attribution required. Cheng et al., Science 2023. Missense variant pathogenicity predictions. Use `--no-alphamissense` to omit. |
+| CADD | cadd.gs.washington.edu | LicenseRef-CADD | Attribution required, **non-commercial by default**. Commercial licenses available from UW CoMotion. Opt-in via `allelix db update --cadd`. |
 
-**Commercial users:** SNPedia content is non-commercial. Set `allelix config set license.commercial true` to permanently disable non-commercial sources, or pass `--exclude-snpedia` per-invocation. Either way, `analyze` runs using all other databases and omits SNPedia annotations automatically. All other databases (ClinVar, PharmGKB, GWAS Catalog, gnomAD, AlphaMissense) are compatible with commercial use.
+**Commercial users:** When `license.commercial = true`, non-commercial sources are gated by a three-state permission model. SNPedia is permanently blocked (no commercial license is available). CADD is blocked by default but can be unlocked — the University of Washington offers commercial licenses at `https://els2.comotion.uw.edu/product/cadd-scores`; after purchasing, assert your license with `allelix config set license.cadd true` to re-enable CADD in commercial mode. All other databases (ClinVar, PharmGKB, GWAS Catalog, gnomAD, AlphaMissense) are compatible with commercial use. `allelix config show` displays the permission state for each source.
 
 ### SNPedia data download
 

SECURITY.md

@@ -6,8 +6,8 @@ Only the latest minor release receives security fixes.
 
 | Version | Supported |
 |---------|-----------|
-| 1.5.x   | ✓         |
-| < 1.5   | ✗         |
+| 1.6.x   | ✓         |
+| < 1.6   | ✗         |
 
 ## Reporting a vulnerability
 

allelix/annotators/__init__.py

@@ -8,6 +8,7 @@
 
 from allelix.annotators.alphamissense import AlphaMissenseAnnotator
 from allelix.annotators.base import Annotator
+from allelix.annotators.cadd import CaddAnnotator
 from allelix.annotators.clinvar import CLINVAR_SUPPORTED_BUILDS, ClinVarAnnotator
 from allelix.annotators.gnomad import GnomadAnnotator
 from allelix.annotators.gwas import GWASCatalogAnnotator
@@ -24,6 +25,7 @@ def get_annotators(
     *,
     include_benign: bool = False,
     gwas_filter_traits: bool = True,
+    cadd_full: bool = False,
 ) -> list[Annotator]:
     """Construct all registered annotators bound to the given data directory.
 
@@ -37,6 +39,9 @@ def get_annotators(
     `gwas_filter_traits` passes through to GWASCatalogAnnotator. Default
     True excludes common-trait noise categories (ADR-0024 amendment).
 
+    `cadd_full` enables CADD full mode (tabix queries against the
+    complete 81 GB CADD file). Requires ``pysam`` and a local copy.
+
     ADR-0023: ClinVar's `reference_for(rsid, build)` is wired into
     PharmGKB and SNPedia as the primary hom-ref suppression filter — the
     REF allele lookup universally determines whether the user is
@@ -48,7 +53,8 @@ def get_annotators(
     snpedia = SNPediaAnnotator(data_dir, clinvar_ref_provider=clinvar.reference_for)
     gnomad = GnomadAnnotator(data_dir)
     alphamissense = AlphaMissenseAnnotator(data_dir)
-    return [clinvar, pharmgkb, gwas, snpedia, gnomad, alphamissense]
+    cadd = CaddAnnotator(data_dir, full_mode=cadd_full)
+    return [clinvar, pharmgkb, gwas, snpedia, gnomad, alphamissense, cadd]
 
 
 _ANNOTATOR_CLASSES: dict[str, type[Annotator]] = {
@@ -60,6 +66,7 @@ def get_annotators(
         SNPediaAnnotator,
         GnomadAnnotator,
         AlphaMissenseAnnotator,
+        CaddAnnotator,
     ]
 }
 
@@ -72,6 +79,7 @@ def get_annotator_class(name: str) -> type[Annotator] | None:
 __all__ = [
     "AlphaMissenseAnnotator",
     "Annotator",
+    "CaddAnnotator",
     "ClinVarAnnotator",
     "GWASCatalogAnnotator",
     "GnomadAnnotator",

allelix/annotators/alphamissense.py

@@ -65,6 +65,7 @@ class AlphaMissenseAnnotator(Annotator):
         ),
         source_url="https://zenodo.org/records/10813168",
         citation="Cheng et al., Science 2023 (doi:10.1126/science.adg7492)",
+        commercial_ok=True,
     )
 
     def __init__(self, data_dir: Path) -> None:

allelix/annotators/base.py

@@ -7,6 +7,7 @@
 import contextlib
 from abc import ABC, abstractmethod
 from dataclasses import dataclass
+from enum import Enum, auto
 from typing import TYPE_CHECKING, ClassVar
 
 if TYPE_CHECKING:
@@ -26,6 +27,9 @@ class LicenseDescriptor:
     attribution_text: str
     source_url: str | None = None
     citation: str | None = None
+    commercial_ok: bool | None = None
+    licensable: bool = False
+    purchase_url: str | None = None
 
 
 _NON_COMMERCIAL_SPDX: frozenset[str] = frozenset(
@@ -37,9 +41,37 @@ class LicenseDescriptor:
 )
 
 
-def is_non_commercial(spdx: str) -> bool:
-    """Return True if the SPDX identifier prohibits commercial use."""
-    return spdx in _NON_COMMERCIAL_SPDX
+def is_non_commercial(descriptor: LicenseDescriptor) -> bool:
+    """Return True if the license prohibits commercial use."""
+    if descriptor.commercial_ok is not None:
+        return not descriptor.commercial_ok
+    return descriptor.spdx in _NON_COMMERCIAL_SPDX
+
+
+class Permission(Enum):
+    """Three-state permission result for a source in the current license context."""
+
+    ALLOW = auto()
+    BLOCK_FINAL = auto()
+    BLOCK_PURCHASABLE = auto()
+
+
+def permission(
+    descriptor: LicenseDescriptor,
+    *,
+    commercial: bool,
+    license_held: bool,
+) -> Permission:
+    """Determine whether a source is permitted under the current license context."""
+    if not commercial:
+        return Permission.ALLOW
+    if not is_non_commercial(descriptor):
+        return Permission.ALLOW
+    if not descriptor.licensable:
+        return Permission.BLOCK_FINAL
+    if license_held:
+        return Permission.ALLOW
+    return Permission.BLOCK_PURCHASABLE
 
 
 def is_clinvar_homref(
@@ -88,6 +120,22 @@ def __init_subclass__(cls, **kwargs: object) -> None:
         if not is_abstract and not hasattr(cls, "license"):
             msg = f"{cls.__name__} must declare a 'license' ClassVar of type LicenseDescriptor"
             raise TypeError(msg)
+        if hasattr(cls, "license"):
+            desc = cls.license
+            if (
+                desc.spdx.startswith("LicenseRef-") or desc.spdx.startswith("custom-")
+            ) and desc.commercial_ok is None:
+                msg = (
+                    f"{cls.__name__} uses custom SPDX '{desc.spdx}' but "
+                    f"does not declare commercial_ok (True or False)"
+                )
+                raise TypeError(msg)
+            if desc.licensable and desc.purchase_url is None:
+                msg = (
+                    f"{cls.__name__} declares licensable=True but "
+                    f"purchase_url is None — set it explicitly"
+                )
+                raise TypeError(msg)
 
     def __init__(self, data_dir: Path) -> None:
         """Bind the annotator to a data directory (created elsewhere)."""