fix(structural): statutory legal floor + TCMB double-count audit; dataset expansion + identification diagnostics

Freeze the structural architecture (no new model/source/quality abstraction/SaleProbability/ML). Two corrections + a dataset-expansion & identification sprint over the already-validated UYAP/KAP/TOKI families.

1) Statutory floor: legal_floor = max(0.5*Q, priority_claims) + realization_costs (was max(0.5Q, pc+rc)). muhammen_bedel preserved as appraised value Q, never the reserve/floor. Missing components -> legal_floor_exact=false with observed components preserved (lower bound, not fabricated).
2) TCMB anchor audit: a contemporaneous provincial TL/m2 level is used directly (V=U_gt*m2*exp(b'dZ)); the full KFE level is NOT multiplied again. roll_unit_price(U_t0,kfe_t,kfe_t0)=U_t0*(kfe_t/kfe_t0) applies the KFE RATIO only when rolling an older anchor. CLI --kfe-factor removed.
- UYAP schema expanded (public_record_id, auction_date, offer_count, source_audited, parcel/unit-net/unit-gross areas kept DISTINCT, never substituted); sold+unsold; uyap_observed_moments adds variance/quantiles/mass-near-floor/offer-bidder distribution.
- KAP structural dataset (real-estate disposal, related_party=false, realized price + appraisal/prior reference); moments over log(sale/appraisal). Not documented as 'KAP gives eta' -- eta jointly estimated with the mechanism shift.
- TOKI guards: same project + same table semantics + non-decreasing count&total; revision_detected on decrease/semantics-change/stratum-disappearance; no property-level pairs.
- Identification: sold structural identify -> dataset counts + numerical moment Jacobian (central differences, common random numbers), rank/singular values/condition number/weak directions, eta+shift profile diagnostics. rank(J)<dim(theta) -> NOT_IDENTIFIED -> prediction switches to sensitivity mode. Optimizer convergence is not identification.
- Terminology: structural value is a 'structural-method prototype using provisional parameters' (sensitivity mode) until identification supports the vector; still 'inferred structural distribution', never observed/measured.
- Validated Level-2 (KAP 963554, TOKI PMVR3, UYAP 16766356960) unchanged; consumer path frozen; no fabricated fields/floors/counts/cohorts/identification.
- 161 tests passing

Author: onatozmenn

README.md

@@ -4,7 +4,7 @@
 [![Data refresh](https://github.com/onatozmenn/sold/actions/workflows/kfe-refresh.yml/badge.svg)](https://github.com/onatozmenn/sold/actions/workflows/kfe-refresh.yml)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
-[![Tests](https://img.shields.io/badge/tests-145%20passing-brightgreen.svg)](tests/)
+[![Tests](https://img.shields.io/badge/tests-161%20passing-brightgreen.svg)](tests/)
 [![Data](https://img.shields.io/badge/data-TCMB%20%C2%B7%20T%C3%9C%C4%B0K-informational.svg)](#data-sources)
 
 > Infer the **realized transaction price** of a Turkish home from its **asking** price — a provenance-aware valuation engine.
@@ -124,12 +124,14 @@ Each public source enters as **structural moments under its own mechanism**, nev
 
 | Source | Structural role |
 |---|---|
-| **TCMB** TL/m² + KFE/YÖKFE | Hedonic **fair-value level anchor** (appraisal, *not* transactions). The log-linear hedonic gives **relative** characteristic premiums only — the listing-price **intercept is never used as the price level** |
-| **UYAP** e-Satış | **Structural auctions** (sold *and* unsold): bidder valuations from the buyer-value distribution vs the **statutory legal floor** `max(0.5·Q, priority_claims + realization_costs)`. `muhammen_bedel = appraised value Q`, **not** the reserve; unobserved floor components stay *partially observed*, never fabricated |
-| **KAP** | Non-related negotiated disposals → **`η` calibration moments** with an explicit **corporate mechanism/domain shift** (never treated as ordinary-resale truth) |
-| **TOKİ** | Repeated cumulative disclosures **differenced within room-type strata** → period realized-sale cohort moments, with reconciliation/revision guards (no property-level pairs, no asking→closing discount) |
+| **TCMB** TL/m² + KFE/YÖKFE | Hedonic **fair-value level anchor** (appraisal, *not* transactions). The log-linear hedonic gives **relative** characteristic premiums only — the listing-price **intercept is never used as the price level**. A **contemporaneous** provincial TL/m² is the level anchor **used directly** (`V = U_gt · m² · e^{β'ΔZ}`); the **KFE ratio** is applied **only** to roll an *older* anchor `U_{t0} · (KFE_t/KFE_{t0})` — a level is never multiplied by the full index again (no temporal double count) |
+| **UYAP** e-Satış | **Structural auctions** (sold *and* unsold): bidder valuations from the buyer-value distribution vs the **statutory legal floor** `max(0.5·Q, priority_claims) + realization_costs`. `muhammen_bedel = appraised value Q`, **not** the reserve; unobserved floor components stay *partially observed* (`legal_floor_exact=false`), never fabricated. Structural observation preserves offer/bidder counts and **distinct** parcel / unit-net / unit-gross areas (never substituted) |
+| **KAP** | Non-related negotiated real-estate disposals → moments over **`log(sale/appraisal)`** that help **jointly** calibrate bargaining power **and** an explicit **corporate mechanism/domain shift** (this is *not* "KAP gives η"; η is jointly estimated, never ordinary-resale truth) |
+| **TOKİ** | Repeated cumulative disclosures **differenced within room-type strata** (same project, same table semantics, non-decreasing count *and* total) → period realized-sale cohort moments; a `revision_detected` guard blocks differencing on decreases, semantics changes or vanished strata (no property-level pairs, no asking→closing discount) |
 
-For an ordinary listing, **asking price is a noisy strategic signal of the seller reservation** (not ground truth, not a ceiling): `S` is conditioned on asking, fair value and tightness, `B`/`S` are drawn, trades (`B ≥ S`) retained, and the **conditional-on-trade** closing distribution returned — median, mean, an **80% structural interval**, trade probability, and **mechanism-transfer sensitivity**. This is a **structural inference, never an observed closing price or a measured ordinary-resale accuracy**. See [`src/sold/structural/`](src/sold/structural/) and `sold structural value` / `sold structural estimate`.
+For an ordinary listing, **asking price is a noisy strategic signal of the seller reservation** (not ground truth, not a ceiling): `S` is conditioned on asking, fair value and tightness, `B`/`S` are drawn, trades (`B ≥ S`) retained, and the **conditional-on-trade** closing distribution returned — median, mean, an **80% structural interval**, trade probability, and **mechanism-transfer sensitivity**. This is a **structural inference, never an observed closing price or a measured ordinary-resale accuracy**.
+
+**Identification before estimation.** Optimizer convergence is *not* identification. `sold structural identify` inspects the actual dataset and computes a numerical moment Jacobian `J(θ)=∂m_sim/∂θ'` (central differences, common random numbers) reporting rank, singular values, condition number and weakly-identified directions, plus per-parameter profile diagnostics for `θ` and the mechanism shifts. If `rank(J) < dim(θ)` the status is **`NOT_IDENTIFIED`** and prediction runs in **sensitivity mode**. Until an identified fit exists, `sold structural value` is labelled a **structural-method prototype using provisional parameters** — not a measured ordinary-resale model. See [`src/sold/structural/`](src/sold/structural/) and `sold structural value` / `identify` / `estimate`.
 
 ## Broker Data Flywheel
 
@@ -299,7 +301,7 @@ tests/               # offline unit / end-to-end tests
 ## Testing
 
 ```bash
-pytest -q             # 145 tests, fully offline (no network or API key required)
+pytest -q             # 161 tests, fully offline (no network or API key required)
 ```
 
 ## Methodology & References
@@ -322,7 +324,8 @@ Negotiation-margin figures from Turkish market reporting: İstanbul ≈ 10%, Ank
 - [x] **Consumer direct-label acquisition path** — self-serve seller collector that turns a completed ordinary home sale into a provenance-aware **direct** label (`domain=consumer` · `seller_self_reported` · `ordinary_resale` · `reference=asking` · confidence `B`) eligible for `asking_to_closing_labels()` while public UYAP/KAP/TOKİ stay excluded; returns immediate non-ML seller analytics (initial/final ask-to-close gap, days to close, price cuts) and an **honest** segment benchmark (no fabricated benchmark when observations are insufficient)
 - [x] **Direct-label quality gate (pre-ML)** — mandatory `origin` (`consumer_submission` / `test_fixture` / `demo_seed` / `manual_import`) so `asking_to_closing_labels()` **excludes test/demo by default** (opt-in `include_non_production=True`) and fixtures never inflate the genuine count; `quality_status` (`accepted`/`flagged`/`rejected`) that **hard-rejects only structurally impossible values** (non-positive price, closing-before-listing) and merely **flags** unusual ratios (extreme close-to-ask, final-above-initial, suspicious duration, duplicate) while preserving the original self-reported values; a privacy-preserving duplicate-candidate **fingerprint** (one-way SHA-256 over bucketed canonical non-personal fields that flags submissions collapsing to the **same canonical transaction fingerprint** — a canonical-fingerprint collision, **not** general near-duplicate similarity detection, and it does **not** identify a property or seller); genuine vs test/demo reported as **separate counts**
 - [ ] **First genuine real-world label** — exactly one *actual* seller-submitted completed residential sale passing through the product path + quality gate. **Current genuine direct-label count: 0** — the end-to-end test proves the acquisition *path* works, not that a real-world label has been acquired
-- [x] **Structural econometric core** — mechanism-aware generalized **Nash bargaining** (`P = ηB+(1−η)S`, `η` estimated, not hard-coded) fit by **Simulated Method of Moments**; TCMB-anchored hedonic fair value (relative premiums only, no listing intercept as level); structural UYAP auctions with the **statutory legal floor** (`muhammen_bedel` preserved as appraised value `Q`, never the reserve; partially-observed floors not fabricated); KAP `η`-calibration moments with a corporate mechanism shift; TOKİ cumulative-disclosure differencing into room-type cohort moments. Replaces weak-label aggregation as the core; the provenance registry and validated KAP/TOKİ/UYAP Level-2 records are kept; the consumer path is frozen as an optional future *validation* channel. **Next: public structural dataset expansion + SMM estimation.** No SaleProbability model yet
+- [x] **Structural econometric core** — mechanism-aware generalized **Nash bargaining** (`P = ηB+(1−η)S`, `η` estimated, not hard-coded) fit by **Simulated Method of Moments**; TCMB-anchored hedonic fair value (relative premiums only, no listing intercept as level); structural UYAP auctions with the **statutory legal floor** (`muhammen_bedel` preserved as appraised value `Q`, never the reserve; partially-observed floors not fabricated); KAP `η`-calibration moments with a corporate mechanism shift; TOKİ cumulative-disclosure differencing into room-type cohort moments. Replaces weak-label aggregation as the core; the provenance registry and validated KAP/TOKİ/UYAP Level-2 records are kept; the consumer path is frozen as an optional future *validation* channel. No SaleProbability model yet
+- [x] **Statutory-floor fix, TCMB double-count audit & identification diagnostics** — corrected the İİK acceptance floor to `max(0.5·Q, priority_claims) + realization_costs`; audited the fair-value anchor so a contemporaneous TL/m² level is not trend-adjusted twice (KFE ratio only when rolling an older anchor); expanded the UYAP/KAP/TOKİ structural dataset schemas + observed-moment constructors (area semantics preserved, `log(sale/appraisal)` KAP moments, revision-guarded TOKİ cohorts); added `sold structural identify` (Jacobian rank / singular values / condition number / weak directions / profile diagnostics → `NOT_IDENTIFIED` → sensitivity mode). **Next: public structural dataset expansion → observed moments → identification → only then real SMM estimation**
 - [ ] **SaleProbability** model (`P(sold ≤ N days)`) trained on collected outcomes
 - [ ] Live, ToS-reviewed fetchers for the public label sources
 - [ ] Broker-vs-benchmark analytics over an aggregate anonymized dataset

src/sold/cli.py

@@ -1391,14 +1391,14 @@ def structural_value_cmd(
     asking: float = typer.Argument(..., help="İlan (asking) fiyatı, TL"),
     province: str = typer.Option("İstanbul", help="İl"),
     gross_m2: float = typer.Option(100.0, help="Brüt m²"),
-    kfe_factor: float = typer.Option(1.0, help="KFE/YÖKFE zaman çarpanı (endeks_t/baz)"),
     tightness: float = typer.Option(0.0, help="Piyasa sıkılığı (TÜİK hacminden; 0=nötr)"),
 ) -> None:
-    """Sıradan ilan için YAPISAL closing dağılımı — gözlenen fiyat DEĞİL.
+    """Sıradan ilan için YAPISAL-YÖNTEM PROTOTİPİ closing dağılımı — gözlenen fiyat DEĞİL.
 
     asking, satıcı rezervasyonuna GÜRÜLTÜLÜ sinyaldir (tavan değil). Fair value TCMB
-    ekspertiz TL/m²'ye çıpalıdır. θ şu an ÖNSEL'dir (SMM ile gerçek veriye kalibre
-    edilmemiştir); çıktı yapısal bir çıkarımdır, ölçülen doğruluk değildir.
+    çağdaş ekspertiz TL/m²'ye çıpalıdır (EK KFE çarpanı YOK — çift-sayım önlenir). θ şu an
+    PROVİZYONEL'dir (kimliklendirme raporu desteklemeden ölçülen model DEĞİL); çıktı
+    sensitivity modunda bir yapısal-yöntem prototipidir.
     """
     from .model.synthetic import load_province_ppm2
     from .structural import (
@@ -1408,31 +1408,32 @@ def structural_value_cmd(
     )
 
     ppm2 = load_province_ppm2().get(province)
-    fv = tcmb_fair_value(ppm2, gross_m2, kfe_factor=kfe_factor)
+    fv = tcmb_fair_value(ppm2, gross_m2)  # çağdaş çıpa; EK KFE ÇARPANI YOK
     if fv is None:
         typer.secho(
             f"{province} için TCMB ekspertiz TL/m² bulunamadı (gross_m2 > 0 olmalı).",
             fg=typer.colors.RED,
         )
         raise typer.Exit(code=1)
 
-    out = StructuralClosingPredictor(StructuralParams()).predict(
+    # identified=False → sensitivity mode (kimliklendirilmiş SMM tahmini yok)
+    out = StructuralClosingPredictor(StructuralParams(), identified=False).predict(
         asking, fv, tightness=tightness
     )
     typer.secho(
-        "YAPISAL closing çıkarımı (θ ÖNSEL — SMM ile kalibre EDİLMEMİŞ)",
+        "YAPISAL-YÖNTEM PROTOTİPİ (θ PROVİZYONEL — kimliklendirilmemiş; sensitivity mode)",
         fg=typer.colors.CYAN,
         bold=True,
     )
-    typer.echo(f"  İlan: {asking:,.0f} TL · Fair value (TCMB ekspertiz çıpalı): {fv:,.0f} TL")
+    typer.echo(f"  İlan: {asking:,.0f} TL · Fair value (TCMB çağdaş ekspertiz çıpası): {fv:,.0f} TL")
     med = out["inferred_closing_median"]
     if med is None:
         typer.secho("  Bu senaryoda ticaret olasılığı ~0.", fg=typer.colors.YELLOW)
     else:
         lo, hi = out["interval_80"]
         typer.echo(f"  Çıkarımsal closing: medyan {med:,.0f} · ortalama {out['inferred_closing_mean']:,.0f} TL")
         typer.echo(f"  %80 yapısal aralık: {lo:,.0f} – {hi:,.0f} TL")
-    typer.echo(f"  Ticaret olasılığı: {out['trade_probability']:.2f}")
+    typer.echo(f"  Ticaret olasılığı: {out['trade_probability']:.2f}  ·  mod: {out['mode']}")
     band = out["mechanism_transfer_sensitivity"].get("median_band")
     if band and band[0] is not None:
         typer.echo(f"  Mekanizma-transfer duyarlılığı (medyan bandı): {band[0]:,.0f} – {band[1]:,.0f} TL")
@@ -1482,6 +1483,114 @@ def structural_estimate_cmd(
     )
 
 
+@structural_app.command("identify")
+def structural_identify_cmd(
+    auctions_file: Optional[Path] = typer.Option(None, "--auctions", help="UYAP açık artırma kayıtları (JSON)"),
+    kap_file: Optional[Path] = typer.Option(None, "--kap", help="KAP elden çıkarma kayıtları (JSON)"),
+    toki_file: Optional[Path] = typer.Option(None, "--toki", help="TOKİ açıklamaları (JSON)"),
+    demo: bool = typer.Option(False, "--demo", help="Sentetik veriyle diagnostiği göster"),
+) -> None:
+    """Yapısal KİMLİKLENDİRME raporu: dataset sayıları + Jacobian rank/SVD/koşul + profiller.
+
+    Optimizer yakınsaması KİMLİKLENDİRME DEĞİLDİR. rank(J) < dim(θ) ise NOT_IDENTIFIED ve
+    tahmin sensitivity moduna geçer. Epistemik katı: veri yoksa 0 raporlanır (uydurma yok).
+    """
+    import json
+
+    import numpy as np
+
+    from .structural import (
+        DEFAULT_FREE,
+        MomentContext,
+        StructuralParams,
+        context_from_datasets,
+        difference_disclosures,
+        identification_report,
+        kap_observed_moments,
+        load_auctions,
+        load_kap_disposals,
+        observed_moments,
+        simulate_negotiations,
+        uyap_observed_moments,
+    )
+
+    auctions_df = kap_df = None
+    toki_res = None
+    m_obs: dict = {}
+
+    if demo:
+        theta0 = StructuralParams(eta=0.6)
+        rng = np.random.default_rng(7)
+        V = np.ones(20000)
+        neg = simulate_negotiations(rng, V, theta0, 20000, mechanism="kap")
+        tr = neg["traded"]
+        m_obs = observed_moments(kap_realized=neg["price"][tr], kap_appraisal=V[tr])
+        ctx = MomentContext(
+            auction_appraised=np.array([]), auction_floors=np.array([]),
+            kap_appraisal=np.ones(60), reps=300,
+        )
+    else:
+        if auctions_file:
+            auctions_df = load_auctions(
+                json.loads(Path(auctions_file).read_text(encoding="utf-8"))
+            )
+            m_obs.update(uyap_observed_moments(auctions_df))
+        if kap_file:
+            kap_df = load_kap_disposals(
+                json.loads(Path(kap_file).read_text(encoding="utf-8"))
+            )
+            m_obs.update(kap_observed_moments(kap_df))
+        if toki_file:
+            toki_res = difference_disclosures(
+                json.loads(Path(toki_file).read_text(encoding="utf-8"))
+            )
+        ctx = context_from_datasets(auctions_df, kap_df)
+
+    rep = identification_report(
+        ctx, StructuralParams(), DEFAULT_FREE, m_obs=m_obs,
+        auctions=auctions_df, kap=kap_df, toki_result=toki_res,
+    )
+    ds = rep["dataset"]
+    typer.secho("Yapısal kimliklendirme raporu", fg=typer.colors.CYAN, bold=True)
+    typer.echo(
+        f"  UYAP: toplam {ds['uyap_total']} · satılan {ds['uyap_sold']} · "
+        f"satılmayan {ds['uyap_unsold']}"
+    )
+    typer.echo(
+        f"        teklif gözlemli {ds['uyap_offer_count_observed']} · artıran gözlemli "
+        f"{ds['uyap_bidder_count_observed']} · tam yasal-taban {ds['uyap_exact_legal_floor_observed']}"
+    )
+    typer.echo(f"  KAP müzakereli elden çıkarma: {ds['kap_negotiated_disposals']}")
+    typer.echo(f"  TOKİ geçerli proje-dönem strata: {ds['toki_valid_project_period_strata']}")
+    typer.echo(
+        f"  Yapısal parametre (dim θ): {rep['n_structural_parameters']} · "
+        f"gözlenen moment: {rep['n_observed_moments']}"
+    )
+    color = typer.colors.GREEN if rep["status"] == "IDENTIFIED" else typer.colors.RED
+    typer.secho(
+        f"  DURUM: {rep['status']}  (rank {rep['rank']} / dim {rep['n_structural_parameters']}) "
+        f"· mod: {rep['prediction_mode']}",
+        fg=color,
+        bold=True,
+    )
+    if rep.get("singular_values"):
+        sv = ", ".join(f"{x:.2e}" for x in rep["singular_values"])
+        typer.echo(f"  Tekil değerler: [{sv}]  ·  koşul sayısı: {rep['condition_number']:.2e}")
+    for wd in rep.get("weakly_identified_directions", []):
+        typer.echo(f"  Zayıf-kimliklendirilmiş yön (s={wd['singular_value']:.2e}): {wd['direction']}")
+    for pname, pr in (rep.get("profiles") or {}).items():
+        rr = pr.get("relative_range")
+        flag = "ZAYIF" if pr.get("weakly_identified") else "belirgin"
+        rr_txt = f"{rr:.2e}" if isinstance(rr, (int, float)) else "—"
+        typer.echo(f"  Profil {pname}: göreli hedef aralığı {rr_txt} → {flag}")
+    if rep["status"] != "IDENTIFIED":
+        typer.secho(
+            "  Not: rank < dim → optimizer sonucu NOKTA TAHMİNİ olarak SUNULMAZ; sensitivity "
+            "mode. (Optimizer yakınsaması ≠ kimliklendirme.) Sonraki iş: veri kümesi genişletmesi.",
+            fg=typer.colors.YELLOW,
+        )
+
+
 @app.command("serve")
 def serve_cmd(
     host: str = typer.Option("127.0.0.1", help="Dinlenecek adres"),