Correct UYAP outcome taxonomy, remove invalid uyap_sale_prob, pivot to PARTIAL IDENTIFICATION + identification-aware prediction

e-Satis exposes four top-level states (Satildi / Birinci Aliciya Sure Verildi / Malin Satisinin Dusmesi / Ihale Sonucu Girilmemistir); no fifth status invented. Only Satildi is a completed sale; settlement-pending/missing-result are CENSORED (not sold=false); Malin Satisinin Dusmesi is reason-dependent (withdrawal/Satistan Vazgecilmesi is administrative, not a market no-trade). classify_auction_outcome + outcome_status/outcome_reason/trade_outcome_class fields; raw taxonomy preserved.

Removed uyap_sale_prob from m_obs, simulated moments and the Jacobian (reason: public UYAP outcome taxonomy does not currently identify a comparable negative auction trade class). Not replaced with 0/1/guessed rate. Kept uyap_win_over_appraisal_mean/sd as conditional-on-completed-sale moments. Measured: m_obs 5->4, but condition number 1.7e17 -> 61.8 (degenerate sale_prob=1.0 removed the spurious ~0 singular value); rank(J_combined) stays 4/dim 6.

New partial.py: admissible_set Theta_I={theta:Q(theta)<=Q_min+tol} with explicit sensitivity-tested tolerance tol=max(1e-4, rel*|Q_min|), common random numbers, reproducible global+local sampling; measured set-identified mu_s/sigma_s/eta, point-like mu_b/sigma_b/auction_shift, parameter trade-off correlations. PartiallyIdentifiedPredictor separates within-theta negotiation uncertainty from between-theta identification uncertainty (status=PARTIALLY_IDENTIFIED); never called observed price or measured accuracy. CLI: structural partial + value --partial. theta NOT shrunk for rank; TOKI stays external; no ML/weak-supervision/SaleProbability/fourth source. 187 tests passing.

Author: onatozmenn

README.md

@@ -4,7 +4,7 @@
 [![Data refresh](https://github.com/onatozmenn/sold/actions/workflows/kfe-refresh.yml/badge.svg)](https://github.com/onatozmenn/sold/actions/workflows/kfe-refresh.yml)
 [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
-[![Tests](https://img.shields.io/badge/tests-182%20passing-brightgreen.svg)](tests/)
+[![Tests](https://img.shields.io/badge/tests-187%20passing-brightgreen.svg)](tests/)
 [![Data](https://img.shields.io/badge/data-TCMB%20%C2%B7%20T%C3%9C%C4%B0K-informational.svg)](#data-sources)
 
 > Infer the **realized transaction price** of a Turkish home from its **asking** price — a provenance-aware valuation engine.
@@ -301,7 +301,7 @@ tests/               # offline unit / end-to-end tests
 ## Testing
 
 ```bash
-pytest -q             # 182 tests, fully offline (no network or API key required)
+pytest -q             # 187 tests, fully offline (no network or API key required)
 ```
 
 ## Methodology & References
@@ -332,6 +332,7 @@ Negotiation-margin figures from Turkish market reporting: İstanbul ≈ 10%, Ank
 - [x] **TOKİ reclassified as `external_cross_mechanism_benchmark` & KAP 265789→312317 audit (PENDING)** — the 5 genuine TOKİ cohort/composition moments are **not** labelled *unavailable* (they are observed and available); they are declared an **external cross-mechanism benchmark** that is deliberately **outside** the current SMM identification system (`moments_used_in_identification = 0`, reason: *no current model-implied primary-market counterpart*). `sold structural identify` reports this explicitly; **`rank(J_TOKİ)` stays 0 and was _not_ “fixed” by inventing a primary-market mechanism, and `θ` was _not_ shrunk to force rank**. Separately, a candidate second KAP negotiated disposal (records `265789→312317`, appraisal 15.487M TRY → sale 8.0M USD +VAT) is recorded under [`validation/structural/kap_candidates.json`](validation/structural/) as **`PENDING_AUDIT`** and **does not enter the genuine set / `kap_log_ratio` moments** — admission is blocked by conditions that cannot be verified without fabrication: same-transaction only *operator-linked* (#1), related-party *unverified* (#4), **currency mismatch** TRY vs USD with **no documented official TCMB rate** (#5,#6), and **VAT non-comparability** (#7,#8). Measured identification is therefore unchanged: `m_obs`=3, `J_UYAP` rank 1 / `J_KAP` rank 1 / `J_combined` rank 2, singular values `[1.73, 0.579, 0]`, condition →∞, **`NOT_IDENTIFIED` (rank 2 / dim 6)**. Still operator-blocked (never fabricated): 1 unsold + 1 additional sold UYAP auction, and an admissible same-currency/VAT-comparable second KAP disposal
 - [x] **Genuine audited second KAP disposal (currency + VAT + related-party source-audited) → `kap_log_ratio_sd` unlocked, rank 2→3** — the previously `PENDING_AUDIT` KAP chain `265789→312317` was manually source-audited against the official KAP disclosure chain (linked as **one** Kapadık/Esenyurt, 382 ada 11 parsel disposal via the previous-notification reference — **not** two transactions), the official valuation-report attachment (`KDV Hariç`), and the official TCMB EVDS series `TP.DK.USD.A.YTL`. Final realized consideration `7,533,161 USD + KDV` (from completion update `312317`, superseding the earlier `8,000,000 USD` **proposal**) normalized to TRY at the documented buying rate `2.03650000` on the completion/conversion date `2013-10-01` → `15,341,282.3765 TRY`; appraisal `15,487,102 TRY`, both compared **excluding VAT** (no VAT added to either side); related-party from the official old-form `Karşı Taraf İle Olan İlişkinin Niteliği = Yoktur` → `related_party=false` (old-form provenance preserved; **no** modern structured SPK boolean invented). `log(sale/appraisal) = −0.009460159132789749`. Admitted as one genuine negotiated-disposal observation (`corporate_negotiated_non_related`; excluded from `asking_to_closing_labels()`). **Measured genuine identification gain: KAP negotiated count 1→2, `kap_log_ratio_sd` unavailable→observable, `m_obs` 3→4, `rank(J_KAP)` 1→2, `rank(J_combined)` 2→3, singular values `[1.706, 0.678, 0.118, 1.3e-18]`, smallest non-zero sv `0.579→0.118`; status stays `NOT_IDENTIFIED` (rank 3 / dim 6) — real data raised the rank without touching θ, TOKİ stays `external_cross_mechanism_benchmark`, and no ML / weak-supervision / fourth-source layer was added.** UYAP still operator-blocked: 1 unsold + 1 additional sold auction
 - [x] **Genuine audited second sold UYAP auction (e-Satış `16662608597`) → `uyap_win_over_appraisal_sd` unlocked, rank 3→4** — manually source-audited from the public e-Satış portal + official auction documents (Ankara/Altındağ dükkan): `Takdir Olunan Değer/Kıymeti = 13,000,000 TRY` kept as `appraised_value = Q` (court-appraised value, **not** the reserve/floor), official result `İhale Bedeli 6,550,000 TRY` on `03/06/2026` with `En Yüksek Teklif Verene Malın İhale Edilmesi` → `sold=true`. `winning_bid/appraised_value = 0.5038461538461538` (auction ratio, **not** an ordinary-resale asking-to-closing discount; `domain=uyap`, `sale_mechanism=auction`, excluded from `asking_to_closing_labels()`). Area semantics preserved and **never interchanged**: `parcel_area_m2=1864.72`, `unit_net_m2=325.00`, `unit_gross_m2=350.00`. `priority_claims`/`realization_costs` not sufficiently observed → `legal_floor_exact=false` (50% of Q is a lower bound, not an exact floor). Privacy boundary: no party/representative/counsel/payment/case-party data transcribed — only non-personal economic/property/public-record fields. **Measured genuine identification gain (θ unchanged): UYAP genuine sold 1→2 (2 sold, 0 unsold, `sale_prob` still degenerate 1.0), `uyap_win_over_appraisal_sd` unavailable→observable (mean `0.7569`, sd `0.2531`), `m_obs` 4→5, `rank(J_UYAP)` 1→2, `rank(J_KAP)` 2, `rank(J_combined)` 3→4, singular values `[1.760, 0.975, 0.214, 0.0285, 1.0e-17]`, smallest non-zero sv `0.118→0.0285`, condition `1.7e17`, weak direction `{sigma_s:0.80, mu_s:0.59, eta:-0.07}`; every currently-modeled moment is now observed (`unavailable_moments` empty) yet status stays `NOT_IDENTIFIED` (rank 4 / dim 6) — the remaining gap is degenerate `sale_prob` + limited variation, not a missing moment. TOKİ stays `external_cross_mechanism_benchmark`; no ML / weak-supervision / fourth-source layer added.** UYAP still operator-blocked: 1 unsold auction (to break the degenerate sale probability)
+- [x] **UYAP outcome-taxonomy correction, invalid `uyap_sale_prob` removed, pivot to PARTIAL IDENTIFICATION + identification-aware prediction** — the actual authenticated e-Satış interface exposes four top-level states (`Satıldı`, `Birinci Alıcıya Süre Verildi`, `Malın Satışının Düşmesi`, `İhale Sonucu Girilmemiştir`); **no fifth status was invented** to unlock a sale probability. Only `Satıldı` is a terminal completed sale; settlement-pending / missing-result are **censored** (not `sold=false`), and `Malın Satışının Düşmesi` is reason-dependent (withdrawal/`Satıştan Vazgeçilmesi` is administrative, **not** a market no-trade). Because the public taxonomy cannot separate a comparable negative auction-trade class, **`uyap_sale_prob` was removed from `m_obs`, the simulated moments, and the Jacobian** (documented reason: *public UYAP outcome taxonomy does not currently identify a comparable negative auction trade class*; raw taxonomy + reason preserved for future research, never replaced with a guessed rate). The conditional moments `uyap_win_over_appraisal_mean/sd` are kept, explicitly interpreted as *winning_bid/appraised_value conditional on an observed completed sale*. **Measured effect: removing the degenerate `sale_prob=1.0` moment dropped `m_obs` 5→4 but improved conditioning — condition number `1.7e17 → 61.8` (the spurious ~0 singular value vanished), `rank(J_combined)` stays 4/dim 6.** The final inference gate pivots from forced point identification to **partial identification** `Θ_I = {θ : Q(θ) ≤ Q_min + tol}` (explicit, sensitivity-tested tolerance `tol = max(1e-4, rel·|Q_min|)`, common random numbers, reproducible sampling) via `sold structural partial` — measured `mu_b`/`sigma_b`/`auction_shift` point-like, `mu_s`/`sigma_s`/`eta` **set-identified**, with parameter trade-off correlations. `sold structural value --partial` produces an **identification-aware** closing range that separates *within-θ negotiation uncertainty* from *between-θ identification uncertainty* (`identification_status = PARTIALLY_IDENTIFIED`), never called an observed price or measured ordinary-resale accuracy. θ was **not** shrunk to recover rank; TOKİ stays external; no ML / weak-supervision / SaleProbability / fourth source added. The public-source UYAP no-trade hunt is closed
 - [ ] **SaleProbability** model (`P(sold ≤ N days)`) trained on collected outcomes
 - [ ] Live, ToS-reviewed fetchers for the public label sources
 - [ ] Broker-vs-benchmark analytics over an aggregate anonymized dataset

src/sold/cli.py

@@ -1392,6 +1392,7 @@ def structural_value_cmd(
     province: str = typer.Option("İstanbul", help="İl"),
     gross_m2: float = typer.Option(100.0, help="Brüt m²"),
     tightness: float = typer.Option(0.0, help="Piyasa sıkılığı (TÜİK hacminden; 0=nötr)"),
+    partial: bool = typer.Option(False, "--partial", help="KİMLİKLENDİRME-FARKINDA: Θ_I kabul edilebilir küme üzerinde aralık"),
 ) -> None:
     """Sıradan ilan için YAPISAL-YÖNTEM PROTOTİPİ closing dağılımı — gözlenen fiyat DEĞİL.
 
@@ -1416,6 +1417,48 @@ def structural_value_cmd(
         )
         raise typer.Exit(code=1)
 
+    if partial:
+        # KİMLİKLENDİRME-FARKINDA: gerçek momentlerden Θ_I kur, her θ için simüle et.
+        from .structural import (
+            PartiallyIdentifiedPredictor,
+            admissible_set,
+            build_observed_moments,
+            context_from_datasets,
+            load_genuine_datasets,
+        )
+
+        g = load_genuine_datasets()
+        built = build_observed_moments(g["uyap"], g["kap"], g["toki_result"])
+        ctx = context_from_datasets(g["uyap"], g["kap"])
+        res = admissible_set(built["moments"], ctx, n_candidates=2500)
+        pred = PartiallyIdentifiedPredictor(res.admissible_params, res.best_params).predict(
+            asking, fv, tightness=tightness
+        )
+        typer.secho(
+            "KİMLİKLENDİRME-FARKINDA YAPISAL ÇIKARIM (PARTIALLY_IDENTIFIED)",
+            fg=typer.colors.CYAN, bold=True,
+        )
+        typer.echo(f"  İlan: {asking:,.0f} TL · Fair value (TCMB çağdaş çıpa): {fv:,.0f} TL")
+        typer.echo(f"  Kabul edilebilir parametre kümesi |Θ_I|: {pred['parameter_set_size']}")
+        ce = pred["central_structural_estimate"]
+        if ce is None:
+            typer.secho("  Bu senaryoda ticaret olasılığı ~0 (Θ_I çoğunda no-trade).", fg=typer.colors.YELLOW)
+        else:
+            wlo, whi = pred["within_model_interval"]
+            blo, bhi = pred["between_theta_median_band"]
+            ilo, ihi = pred["identification_aware_lower"], pred["identification_aware_upper"]
+            typer.echo(f"  Merkezi yapısal tahmin: {ce:,.0f} TL")
+            typer.echo(f"  within-θ (pazarlık belirsizliği) %80: {wlo:,.0f} – {whi:,.0f} TL")
+            typer.echo(f"  between-θ (kimliklendirme belirsizliği) medyan bandı: {blo:,.0f} – {bhi:,.0f} TL")
+            typer.secho(
+                f"  KİMLİKLENDİRME-FARKINDA ARALIK (iki belirsizlik birlikte): {ilo:,.0f} – {ihi:,.0f} TL",
+                fg=typer.colors.GREEN,
+            )
+            tlo, thi = pred["trade_probability_band"]
+            typer.echo(f"  Ticaret olasılığı bandı: {tlo:.3f} – {thi:.3f}")
+        typer.secho(f"  {pred['note']}", fg=typer.colors.YELLOW)
+        return
+
     # identified=False → sensitivity mode (kimliklendirilmiş SMM tahmini yok)
     out = StructuralClosingPredictor(StructuralParams(), identified=False).predict(
         asking, fv, tightness=tightness
@@ -1576,6 +1619,7 @@ def structural_identify_cmd(
     m_obs: dict = {}
     provenance: dict = {}
     unavailable: list = []
+    ineligible: list = []
     sample_sizes: dict = {}
 
     if demo:
@@ -1603,13 +1647,15 @@ def structural_identify_cmd(
             auctions_df, kap_df, toki_res = genuine["uyap"], genuine["kap"], genuine["toki_result"]
         built = build_observed_moments(auctions_df, kap_df, toki_res)
         m_obs, provenance, unavailable = built["moments"], built["provenance"], built["unavailable"]
+        ineligible = built.get("ineligible", [])
         sample_sizes = built["sample_sizes"]
         ctx = context_from_datasets(auctions_df, kap_df)
 
     rep = identification_report(
         ctx, StructuralParams(), DEFAULT_FREE, m_obs=m_obs,
         auctions=auctions_df, kap=kap_df, toki_result=toki_res,
-        provenance=provenance, unavailable=unavailable, sample_sizes=sample_sizes,
+        provenance=provenance, unavailable=unavailable, ineligible=ineligible,
+        sample_sizes=sample_sizes,
     )
     ds = rep["dataset"]
     typer.secho("Yapısal kimliklendirme raporu", fg=typer.colors.CYAN, bold=True)
@@ -1627,6 +1673,12 @@ def structural_identify_cmd(
         typer.echo(f"  Moment provenance: {rep['moment_provenance']}")
     for un in rep.get("unavailable_moments", []):
         typer.echo(f"  Eksik moment: {un['moment']} [{un['source']}] — {un['reason']}")
+    for il in rep.get("ineligible_moments", []):
+        typer.secho(
+            f"  KALDIRILDI (SMM DIŞI) moment: {il['moment']} [{il['source']}] "
+            f"(gözlenen={il.get('observed_value')}) — {il['reason']}",
+            fg=typer.colors.YELLOW,
+        )
     # Dış çapraz-mekanizma benchmark (gözlenir + kullanılabilir ama SMM DİŞI — unavailable DEĞİL)
     for fam, eb in (rep.get("external_benchmarks") or {}).items():
         typer.secho(
@@ -1690,13 +1742,70 @@ def structural_identify_cmd(
             typer.echo(f"    durum: {cmp['status']['before']} → {cmp['status']['after']}")
     if rep["status"] != "IDENTIFIED":
         typer.secho(
-            "  Not: rank < dim veya zayıf kimliklendirme → optimizer sonucu NOKTA TAHMİNİ olarak "
-            "SUNULMAZ; sensitivity/prototip mode. (Optimizer yakınsaması ≠ kimliklendirme.) "
-            "Sonraki iş: gerçek UYAP/KAP/tekrarlı TOKİ gözlem genişletmesi.",
+            "  Not: rank < dim → optimizer sonucu tek NOKTA TAHMİNİ olarak SUNULMAZ. Ama artık "
+            "rank(J)=dim ZORUNLU değil: KISMİ KİMLİKLENDİRME (partial identification) kapısı "
+            "devrede — 'sold structural partial' ile kabul edilebilir yapısal parametre kümesi "
+            "Θ_I ve 'sold structural value --partial' ile kimliklendirme-farkında aralık üretilir.",
             fg=typer.colors.YELLOW,
         )
 
 
+@structural_app.command("partial")
+def structural_partial_cmd(
+    candidates: int = typer.Option(3000, "--candidates", help="Örneklenecek aday parametre vektörü sayısı"),
+    rel: float = typer.Option(0.25, "--rel", help="Tolerans göreli payı: tol=max(1e-4, rel·|Q_min|)"),
+    seed: int = typer.Option(12345, "--seed", help="Yeniden-üretilebilir örnekleme tohumu"),
+    sensitivity: bool = typer.Option(True, "--sensitivity/--no-sensitivity", help="Tolerans duyarlılık taraması"),
+) -> None:
+    """KISMİ KİMLİKLENDİRME: kabul edilebilir yapısal parametre kümesi Θ_I ve tanılamaları.
+
+    Θ_I = {θ : Q(θ) ≤ Q_min + tol}. Tolerans AÇIK (tol=max(1e-4, rel·|Q_min|)) ve duyarlılık
+    testli — gizlice dar aralık için seçilmez. Ortak rastgele sayılar; yeniden-üretilebilir
+    örnekleme. Hangi parametrelerin NOKTA yerine KÜME-kimliklendirildiği raporlanır.
+    """
+    from .structural import (
+        admissible_set,
+        build_observed_moments,
+        context_from_datasets,
+        load_genuine_datasets,
+        tolerance_sensitivity,
+    )
+
+    g = load_genuine_datasets()
+    built = build_observed_moments(g["uyap"], g["kap"], g["toki_result"])
+    ctx = context_from_datasets(g["uyap"], g["kap"])
+    res = admissible_set(built["moments"], ctx, n_candidates=candidates, seed=seed, rel=rel)
+
+    typer.secho("Kısmi kimliklendirme (partial identification) — Θ_I", fg=typer.colors.CYAN, bold=True)
+    typer.echo(f"  En iyi SMM hedefi (Q_min): {res.best_objective:.4e}")
+    typer.echo(f"  Tolerans: {res.tolerance:.4e}  ({res.tolerance_rule})")
+    typer.echo(f"  Aday: {res.n_candidates} · kabul edilebilir |Θ_I|: {res.n_admissible}")
+    typer.echo(f"  KÜME-kimliklendirilmiş (set-identified): {res.set_identified() or '—'}")
+    typer.echo(f"  Nokta-benzeri (point-like): {res.point_like() or '—'}")
+    for p, r in res.param_ranges.items():
+        typer.echo(
+            f"    {p}: [{r['min']:.3f}, {r['max']:.3f}] "
+            f"(bant payı {r['range_fraction']:.2f}) → {r['classification']}"
+        )
+    if res.correlations:
+        strong = {k: round(v, 2) for k, v in res.correlations.items() if abs(v) >= 0.5}
+        typer.echo(f"  Güçlü parametre ödünleşimleri (|corr|≥0.5): {strong or '—'}")
+    if sensitivity:
+        typer.secho("  Tolerans duyarlılığı (gizli seçim YOK):", fg=typer.colors.CYAN)
+        for row in tolerance_sensitivity(built["moments"], ctx, rel_grid=(0.10, 0.25, 0.50, 1.00),
+                                         n_candidates=max(candidates // 2, 800), seed=seed):
+            typer.echo(
+                f"    rel={row['rel']:.2f} → tol={row['tolerance']:.3e} · "
+                f"|Θ_I|={row['n_admissible']} · set-id={row['set_identified'] or '—'}"
+            )
+    typer.secho(
+        "  DURUM: PARTIALLY_IDENTIFIED — rank(J)<dim; θ nokta olarak tanımlanmaz. Bu kabul "
+        "edilebilir küme dürüst yapısal belirsizliği taşır (θ rank için KÜÇÜLTÜLMEDİ).",
+        fg=typer.colors.YELLOW,
+        bold=True,
+    )
+
+
 @app.command("serve")
 def serve_cmd(
     host: str = typer.Option("127.0.0.1", help="Dinlenecek adres"),

src/sold/structural/__init__.py

@@ -15,7 +15,9 @@
 
 from .auction import (
     AUCTION_FIELDS,
+    UYAP_OUTCOME_STATUSES,
     auction_moments,
+    classify_auction_outcome,
     legal_floor,
     load_auctions,
     normalize_auction,
@@ -65,7 +67,15 @@
     simulated_moments,
 )
 from .params import DEFAULT_FREE, StructuralParams
-from .predict import StructuralClosingPredictor
+from .partial import (
+    PARAM_BOUNDS,
+    PartialIdentificationResult,
+    admissible_set,
+    identification_tolerance,
+    objective_value,
+    tolerance_sensitivity,
+)
+from .predict import PartiallyIdentifiedPredictor, StructuralClosingPredictor
 from .smm import SMMResult, estimate_smm, nelder_mead, smm_objective
 from .toki import difference_disclosures, toki_composition_moments
 
@@ -90,6 +100,8 @@
     "simulate_auctions",
     "auction_moments",
     "uyap_observed_moments",
+    "classify_auction_outcome",
+    "UYAP_OUTCOME_STATUSES",
     "AUCTION_FIELDS",
     # KAP yapısal veri kümesi
     "normalize_kap_disposal",
@@ -128,6 +140,14 @@
     "save_snapshot",
     "load_snapshot",
     "compare_snapshots",
+    # kısmi kimliklendirme (partial identification)
+    "admissible_set",
+    "PartialIdentificationResult",
+    "PARAM_BOUNDS",
+    "identification_tolerance",
+    "tolerance_sensitivity",
+    "objective_value",
     # tahmin
     "StructuralClosingPredictor",
+    "PartiallyIdentifiedPredictor",
 ]