feat: expand fail-closed KYC document OCR

Author: piyush-gambhir

.env.deploy.example

@@ -16,3 +16,4 @@ DOCKERHUB_TOKEN=
 # GCP_REGION=asia-south1
 # ALLOW_UNAUTHENTICATED=false           # opt in only behind your own auth layer
 # DOCUMENT_OCR_API_TOKEN=               # optional Bearer token for /scan
+# DOCUMENT_OCR_KYC_LANGS=en,devanagari  # optional KYC-only OCR model passes

.github/workflows/test.yml

@@ -1,6 +1,8 @@
 name: Test
 
 on:
+  push:
+    branches: [main]
   pull_request:
     branches: [main]
 
@@ -22,9 +24,6 @@ jobs:
       - name: Run tests
         run: uv run pytest tests/python -v
 
-      - name: Run benchmark
-        run: uv run python benchmarks/accuracy.py
-
   typescript:
     runs-on: ubuntu-latest
     steps:

CHANGELOG.md

@@ -1,5 +1,58 @@
 # Changelog
 
+## 3.0.0 — 2026-07-27
+
+### Non-passport document coverage
+
+- Added conservative MGNREGA/NREGA job-card extraction for household,
+  registration/validity, administrative-location, BPL/category, family, and
+  repeating adult-member fields across English and selected Indian-script
+  label variants.
+- Added conservative National Population Register name-and-address-letter
+  extraction for labelled resident, address, pincode, reference, and issue-date
+  fields. Bare `NPR` acronyms are rejected without recognized issuer context.
+- Added automatic classification, pipeline result blocks
+  (`nregaJobCardFields`, `nprLetterFields`), and matching TypeScript contracts.
+
+### Accuracy and failure semantics
+
+- Non-passport scans now require a document-specific minimum field set and a
+  conservative per-geometry OCR confidence gate before returning `success`;
+  low-confidence regions are excluded before extraction, and partial or
+  low-confidence results return `failure` with structured errors. The bundled
+  HTTP server therefore returns `422` instead of `200` for partial KYC
+  extractions.
+- Added `identifierValid` for offline format/checksum results, semantic
+  date/age checks, and conservative confidence caps. These checks do not claim
+  document authenticity.
+- Non-passport responses no longer expose raw classifier OCR through
+  `probeText`, preventing secondary identifiers from leaking outside the
+  extractor's documented field contract.
+- Added optional KYC-only multi-model recognition through
+  `DOCUMENT_OCR_KYC_LANGS`, leaving passport OCR model selection unchanged.
+  Distinct script readings for the same text box are preserved, and readiness
+  initializes all configured models. Invalid model names fail closed, and the
+  npm local server now waits for `/ready` rather than liveness before accepting
+  scans.
+- Added a private KYC accuracy evaluator with a versioned manifest schema,
+  per-document/per-field and complete-record metrics, false-success gates, and
+  global plus document-specific design/year/issuer/language/capture-quality
+  slice gates. Required per-document variants must have accepted-positive
+  samples before evaluation can run.
+
+Dedicated passport OCR modules (page classification, MRZ parsing, back-page
+extraction, and passport validation) and passport-positive routing are unchanged
+in this release. KYC OCR runs only when the passport probe is unknown or finds
+no targeted text.
+
+### Migration
+
+- Consumers that construct `DocumentScanResult` values must add
+  `identifierValid`, `missingRequiredFields`, `nregaJobCardFields`, and
+  `nprLetterFields`.
+- Treat HTTP `422` as a completed, fail-closed scan result and inspect its
+  structured body; the JavaScript client already does this.
+
 ## 1.2.0 — 2026-06-07
 
 ### Multi-document support (additive, backward-compatible)

CONTRIBUTING.md

@@ -27,7 +27,10 @@ fixtures must be generated, contain fictitious data, and carry a prominent
 snapshots, logs, or commit messages.
 
 Private accuracy datasets can be placed in `benchmark-data/`, which is ignored
-by Git.
+by Git. Non-passport datasets must follow
+[`benchmarks/KYC_DATASET.md`](benchmarks/KYC_DATASET.md): use the versioned
+manifest, record design/year/language/capture-quality slices, and keep the
+locked release split separate from tuning data.
 
 ## Pull requests
 

Makefile

@@ -1,4 +1,4 @@
-.PHONY: dev test test-py test-ts build benchmark docker-build install sync
+.PHONY: dev test test-py test-ts build benchmark benchmark-kyc docker-build install sync
 
 install:
 	uv sync
@@ -21,6 +21,10 @@ build:
 benchmark:
 	uv run python benchmarks/accuracy.py
 
+benchmark-kyc:
+	@test -n "$(KYC_MANIFEST)" || (echo "Set KYC_MANIFEST=/secure/path/manifest.json"; exit 2)
+	uv run python benchmarks/kyc_accuracy.py --manifest "$(KYC_MANIFEST)" $(if $(KYC_DATASET_ROOT),--dataset-root "$(KYC_DATASET_ROOT)",) $(if $(KYC_REPORT),--output "$(KYC_REPORT)",)
+
 sync:
 	cd packages/passport-ocr && bash scripts/sync-python.sh
 

README.md

@@ -4,7 +4,8 @@ Local-first OCR pipeline for passports and Indian KYC documents. It
 preprocesses scans, classifies the document, runs targeted OCR with RapidOCR
 (PP-OCRv5), and extracts structured fields—including passport MRZ data, Indian
 passport back-page fields, and identifier/holder fields for PAN, Aadhaar,
-driving licences, and voter IDs.
+driving licences, voter IDs, MGNREGA/NREGA job cards, and National Population
+Register (NPR) name-and-address letters.
 
 Ships as a Python package with a FastAPI server, plus an npm wrapper at [`packages/passport-ocr`](packages/passport-ocr) that auto-spawns the Python server for Node.js consumers.
 
@@ -24,8 +25,28 @@ Ships as a Python package with a FastAPI server, plus an npm wrapper at [`packag
 | Aadhaar (front/back) | Aadhaar no. (+ VID, masked-card support), name, DOB/YOB, gender, address, pincode | Verhoeff checksum |
 | Driving licence | DL no., name, DOB, issue/validity dates (NT + TR), address, blood group, vehicle class | DL format |
 | Voter ID (EPIC) | EPIC no., name, relation name + type, gender, DOB/age | EPIC format |
+| MGNREGA/NREGA job card | job-card no., household head, registration/validity, location, category/BPL, adult members | Conservative hierarchical format |
+| NPR name/address letter | reference no., resident name, address, pincode, issue date | Not offline-verifiable |
 
-The document type is detected automatically; `/scan` returns the matching field block (`fields`/`backPageFields` for passports, `panFields`/`aadhaarFields`/`drivingLicenceFields`/`voterIdFields` for the others) keyed by `documentType`.
+Together with the existing passport path, this covers the officially valid
+document categories listed in the
+[RBI KYC Master Direction](https://www.rbi.org.in/Scripts/BS_ViewMasDirections.aspx?id=11566);
+PAN is supported as a separate tax identifier.
+
+The document type is detected automatically; `/scan` returns the matching
+field block (`fields`/`backPageFields` for passports, or `panFields`,
+`aadhaarFields`, `drivingLicenceFields`, `voterIdFields`,
+`nregaJobCardFields`, or `nprLetterFields`) keyed by `documentType`.
+
+NREGA and NPR support is experimental until measured against a representative
+private image dataset. The extraction layer handles multiple label/layout
+variants, but deterministic text-region tests are not evidence of real-image
+accuracy.
+
+To keep passport OCR behavior unchanged, a positive passport-page probe is
+never overridden by the KYC router. A driving licence or voter card whose crop
+contains multiple passport-like labels can therefore remain ambiguous; an
+explicit KYC-only entry point is tracked as future work.
 
 ## Quickstart
 
@@ -86,14 +107,16 @@ The package auto-creates a `.venv`, installs the Python deps, and manages the lo
 internally. Set `DOCUMENT_OCR_API_TOKEN` to require
 `Authorization: Bearer <token>` on `/scan`. For internet-facing deployments,
 use platform IAM or an API gateway in addition to application-level controls.
+Incomplete or semantically invalid non-passport extractions return HTTP `422`
+with the full structured failure result.
 
 ## Output
 
 ```jsonc
 {
   "status": "success",                  // success | failure | unsupported_page
-  "documentType": "passport",           // passport | pan | aadhaar | driving_licence | voter_id | unknown
-  "pageType": "passport_biodata",       // passport_biodata | passport_non_biodata | pan | aadhaar | driving_licence | voter_id | unknown
+  "documentType": "passport",           // passport | pan | aadhaar | driving_licence | voter_id | nrega_job_card | npr_letter | unknown
+  "pageType": "passport_biodata",       // same document values, plus passport_biodata | passport_non_biodata | unknown
   "confidence": 0.91,
   "fields": {
     "surname": "...", "givenNames": "...", "fullName": "...",
@@ -119,8 +142,18 @@ use platform IAM or an API gateway in addition to application-level controls.
                             "classOfVehicle": "...", "validityDateTransport": null },
   "voterIdFields":        { "epicNumber": "...", "name": "...", "relationName": "...", "relationType": "father",
                             "gender": "...", "dateOfBirth": "...", "age": null },
+  "nregaJobCardFields":   { "jobCardNumber": "...", "headOfHousehold": "...", "category": "SC",
+                            "registrationDate": "...", "validityFrom": "...", "validityTo": "...",
+                            "address": "...", "village": "...", "gramPanchayat": "...", "block": "...",
+                            "district": "...", "state": "...", "bplStatus": true, "familyId": "...",
+                            "members": [{ "serialNumber": "1", "name": "...",
+                                          "fatherOrHusbandName": "...", "gender": "FEMALE", "age": 39 }] },
+  "nprLetterFields":      { "referenceNumber": "...", "name": "...", "address": "...",
+                            "pincode": "...", "issueDate": "..." },
   "mrzRaw": ["P<IND...", "..."], "mrzValid": true,
   "lowConfidence": false,
+  "identifierValid": null,              // format/checksum only; never an authenticity verdict
+  "missingRequiredFields": [],
   "errors": [], "warnings": [],
   "processingMs": 412
 }
@@ -131,7 +164,37 @@ use platform IAM or an API gateway in addition to application-level controls.
 1. `preprocess` — orientation, document boundary detection, perspective correction, quality checks
 2. `classify_passport_page` — biodata vs non-biodata vs not-a-passport (cheap bottom-crop probe)
 3. passport path: `run_ocr` (RapidOCR PP-OCRv5, full-page fallback when MRZ is missing) → `parse_mrz` (TD3 MRZ with per-field + overall checksum validation) → `extract_back_page` (bilingual label-aware extraction) → `validate` (cross-checks MRZ vs visual fields, computes confidence)
-4. non-passport path: `classify_document` routes full-page OCR to the matching extractor (`pan` / `aadhaar` / `driving_licence` / `voter_id`), validating each document's identifier (PAN format, Verhoeff for Aadhaar, EPIC/DL format)
+4. non-passport path: `classify_document` routes full-page OCR to the matching
+   extractor (`pan` / `aadhaar` / `driving_licence` / `voter_id` /
+   `nrega_job_card` / `npr_letter`), checks minimum required fields, validates
+   identifiers where possible, and fails closed on partial or semantically
+   implausible records
+
+For non-passport documents, `status: "success"` means the extractor returned
+the document-specific minimum field set and passed a conservative OCR-region
+confidence gate. Alternate model readings for the same detected geometry count
+once. `identifierValid` means only an offline format/checksum check passed. NPR
+references have no universal public checksum, so this value is `null`; it is
+never evidence of authenticity.
+
+### Non-passport OCR languages
+
+The default recognition behavior is unchanged (`en`, with the existing
+automatic fallback). If the expected KYC population uses a known script, run
+up to four recognition passes only on the non-passport path:
+
+```bash
+DOCUMENT_OCR_KYC_LANGS=en,devanagari make dev
+```
+
+Available values are `en`, `latin`, `devanagari`, `ka` (Kannada), `ta`
+(Tamil), and `te` (Telugu). Extra models increase latency and memory. The
+bundled RapidOCR version has no Bengali recognition model: Bengali-labelled
+extractor fixtures prove parsing behavior only, not Bengali image OCR. Select
+languages and release thresholds from measured benchmark slices. Server
+readiness initializes every configured model so model-download or startup
+failures are reported before the first scan. Unsupported names or more than
+four configured models fail readiness instead of silently falling back.
 
 Single entry point: `core.pipeline.scan(image_input)`.
 
@@ -183,15 +246,32 @@ The per-document extractors are covered by deterministic `TextRegion` fixtures
 under `tests/python/test_*_extractor.py`. These tests verify parsing and
 validation behavior; they are not a claim of real-world OCR accuracy.
 
-For image-level evaluation, place a private dataset and `manifest.json` under
-`benchmark-data/` and run:
+For the legacy passport image benchmark, place its private dataset and
+`manifest.json` under `benchmark-data/` and run:
 
 ```bash
 make benchmark
 ```
 
-The directory is ignored by Git. Never commit identity documents or personal
-data. See [CONTRIBUTING.md](CONTRIBUTING.md) for fixture rules.
+For a non-passport KYC dataset, use the versioned manifest and release gates:
+
+```bash
+make benchmark-kyc \
+  KYC_MANIFEST=/secure/kyc-eval/manifest.json \
+  KYC_DATASET_ROOT=/secure/kyc-eval \
+  KYC_REPORT=/secure/kyc-eval/reports/main.json
+```
+
+The KYC evaluator reports classification, acceptance/false-success,
+exact/normalized field, complete-record, runtime, per-document, and
+design/year/issuer/language/capture-quality slice metrics without copying
+ground-truth values into its report. Required per-document slices fail manifest
+validation when a declared variant is absent. See
+[`benchmarks/KYC_DATASET.md`](benchmarks/KYC_DATASET.md) for the variant matrix
+and annotation workflow.
+
+`benchmark-data/` is ignored by Git. Never commit identity documents or
+personal data. See [CONTRIBUTING.md](CONTRIBUTING.md) for fixture rules.
 
 ## Privacy and security
 

TODOS.md

@@ -10,35 +10,53 @@ TypeScript SDK tests, back-page accuracy, Cloud Run deploy) were completed in
    anonymized documents or unmistakably synthetic specimens for the private
    `benchmark-data/` suite. Do not commit identity documents to this repository.
 
-2. **End-to-end image benchmark for the new document types.** The PAN / Aadhaar /
-   driving-licence / voter-ID extractors are covered only by deterministic
-   `TextRegion` fixtures (`tests/python/test_*_extractor.py`) — they never run
-   real OCR. Add an image-level benchmark (anonymized real cards with ground-truth
-   labels) to measure true end-to-end accuracy.
-
-3. **Deskew in the preprocessor.** `core/preprocessor.py` does document-boundary
+2. **Populate and lock the KYC image benchmark.** The versioned evaluator,
+   manifest schema, variant slices, and release gates now exist, but the
+   repository intentionally contains no identity-document images. Build the
+   consented private dataset, establish baselines for all six non-passport
+   document types, and require the locked split before release.
+
+3. **Regional-script OCR coverage.** KYC-only model selection can now be
+   configured with `DOCUMENT_OCR_KYC_LANGS`, but automatic selection and
+   Bengali recognition are unavailable. Measure every configured language
+   slice and add models only where the locked dataset demonstrates a gain.
+
+4. **Deskew in the preprocessor.** `core/preprocessor.py` does document-boundary
    perspective correction but no text-line deskew, so rotated/tilted inputs shift
    the spatial label→value relationships the extractors rely on. A Hough /
    projection-profile deskew step would harden real-world phone-photo accuracy.
 
-4. **Passport probe over-eagerly claims KYC cards.** The cheap bottom-crop
-   passport probe (`core/pipeline._extract_targeted_regions` + `classify_passport_page`)
-   treats "SEX" and "DATE OF BIRTH" as biodata hints, so a voter/DL card showing
-   those exact labels in its lower ~55% can mis-route to the passport path before
-   `classify_document` runs. Fix: require an MRZ (or a passport keyword) before
-   committing to the passport path, or run `classify_document` first and only
-   treat as passport when it agrees.
+5. **Multi-page non-passport documents.** PDF preprocessing currently evaluates
+   the first page. Add a KYC-only page aggregation contract before claiming
+   complete support for multi-page NPR letters or job-card continuations.
+
+6. **Explicit KYC routing for ambiguous documents.** To preserve the existing
+   passport-positive path, `scan()` does not override a positive passport-page
+   classification. Some driving licences and voter cards containing multiple
+   passport-like labels can therefore remain on the passport path. Add a
+   separate KYC-only entry point or independent router without changing
+   passport OCR behavior.
 
-5. **Driving licence layout variance.** The DL extractor is best-effort; layouts
+7. **Driving licence layout variance.** The DL extractor is best-effort; layouts
    differ substantially by issuing state. Gather fixtures from more states
    (especially Smart Card DLs and the newer Parivahan format) and tune
    `core/driving_licence_extractor.py`. The DL identifier format check in
    `core/validators.py` is also loose — tighten per-state if needed.
 
-6. **Aadhaar name detection.** Aadhaar has no Latin label for the holder name, so
+8. **Aadhaar name detection.** Aadhaar has no Latin label for the holder name, so
    `_find_name` infers it spatially relative to the DOB line. Validate against
    more real layouts (vertical/horizontal cards, masked Aadhaar, mAadhaar PDF).
 
-7. **SDK retry semantics for 4xx.** Thread structured HTTP status information
+9. **SDK retry semantics for 4xx.** Thread structured HTTP status information
    through retry handling rather than inferring retryability from error-message
    text.
+
+## Completed in 3.0.0
+
+- Added NREGA job-card and NPR-letter extraction, classification, public result
+  contracts, and deterministic variation tests.
+- Added fail-closed non-passport completeness/identifier/semantic assessment.
+- Added the private KYC benchmark framework with versioned manifests,
+  per-document/per-field metrics, variation slices, and threshold gates.
+- Kept dedicated passport modules and passport-positive routing unchanged; new
+  KYC families use only the existing unknown/no-text routing boundary.