Skip to content

Commit 9b697b8

Browse files
committed
Final productization: honest structural product surface over the frozen core (UI + machine-readable API + Model Evidence + Method)
New src/sold/api/structural_product.py assembly layer (calls the FROZEN core only; no new econometrics). Θ_A built once and cached; structural_valuation/model_evidence/method_overview. New endpoints: POST /structural/valuate (machine-readable: methodology=structural_econometrics, identification_status=STRUCTURALLY_UNDERIDENTIFIED, coverage_claim=null, central_structural_estimate, within_theta_negotiation_interval, between_theta_near_fit_band, structural_sensitivity_range, ask_to_fair_value_ratio, input_conflict+warning, genuine 2/2/5, jacobian_rank/parameter_dimension/near_fit_parameter_count; NEVER confidence_interval/accuracy), GET /structural/evidence, GET /structural/method. New single-page UI (3 tabs Değerle/Model Evidence/Method): primary action 'Estimate inferred transaction outcome' (never 'Predict actual sale price'); result hierarchy central -> within-θ -> between-θ -> structural_sensitivity_range with the visible 'not a confidence interval / no frequentist coverage' statement (not a tooltip) -> identification status rank 4/6 in plain language; input_conflict warning with ratio + six CANDIDATE explanations (never auto-assigned, never clamped); one permanent disclaimer. Model Evidence reports genuine UYAP/KAP/TOKİ (2/2/5) honestly and does NOT show test counts as evidence; Method explains anchor->moments->SMM->Θ_A->trade iff B>=S, P=eta*B+(1-eta)*S, with eta NOT measured from KAP and appraised value NOT the auction reserve; [0.5,2.0] bounds documented as configurable product diagnostics. Legacy /valuate etc. preserved. Test-backed correctness fix (predict.py): IdentificationAwarePredictor no longer collapses the whole envelope to null when only the single best-fit θ trades zero — it builds the sensitivity envelope from the near-fit configurations that actually trade, while still returning an honest null when NO admissible θ trades. Θ_A construction / SMM / moments unchanged. 5 API integration tests + 1 predictor regression test. All 192 prior tests preserved; 198 passing. No ML/weak-supervision/SaleProbability/fourth source/new mechanism/auth/billing.
1 parent 0bb7361 commit 9b697b8

6 files changed

Lines changed: 551 additions & 101 deletions

File tree

README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
[![Data refresh](https://github.com/onatozmenn/sold/actions/workflows/kfe-refresh.yml/badge.svg)](https://github.com/onatozmenn/sold/actions/workflows/kfe-refresh.yml)
55
[![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
66
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7-
[![Tests](https://img.shields.io/badge/tests-192%20passing-brightgreen.svg)](tests/)
7+
[![Tests](https://img.shields.io/badge/tests-198%20passing-brightgreen.svg)](tests/)
88
[![Data](https://img.shields.io/badge/data-TCMB%20%C2%B7%20T%C3%9C%C4%B0K-informational.svg)](#data-sources)
99

1010
> Infer the **realized transaction price** of a Turkish home from its **asking** price — a provenance-aware valuation engine.
@@ -301,7 +301,7 @@ tests/ # offline unit / end-to-end tests
301301
## Testing
302302

303303
```bash
304-
pytest -q # 192 tests, fully offline (no network or API key required)
304+
pytest -q # 198 tests, fully offline (no network or API key required)
305305
```
306306

307307
## Methodology & References
@@ -334,6 +334,7 @@ Negotiation-margin figures from Turkish market reporting: İstanbul ≈ 10%, Ank
334334
- [x] **Genuine audited second sold UYAP auction (e-Satış `16662608597`) → `uyap_win_over_appraisal_sd` unlocked, rank 3→4** — manually source-audited from the public e-Satış portal + official auction documents (Ankara/Altındağ dükkan): `Takdir Olunan Değer/Kıymeti = 13,000,000 TRY` kept as `appraised_value = Q` (court-appraised value, **not** the reserve/floor), official result `İhale Bedeli 6,550,000 TRY` on `03/06/2026` with `En Yüksek Teklif Verene Malın İhale Edilmesi` → `sold=true`. `winning_bid/appraised_value = 0.5038461538461538` (auction ratio, **not** an ordinary-resale asking-to-closing discount; `domain=uyap`, `sale_mechanism=auction`, excluded from `asking_to_closing_labels()`). Area semantics preserved and **never interchanged**: `parcel_area_m2=1864.72`, `unit_net_m2=325.00`, `unit_gross_m2=350.00`. `priority_claims`/`realization_costs` not sufficiently observed → `legal_floor_exact=false` (50% of Q is a lower bound, not an exact floor). Privacy boundary: no party/representative/counsel/payment/case-party data transcribed — only non-personal economic/property/public-record fields. **Measured genuine identification gain (θ unchanged): UYAP genuine sold 1→2 (2 sold, 0 unsold, `sale_prob` still degenerate 1.0), `uyap_win_over_appraisal_sd` unavailable→observable (mean `0.7569`, sd `0.2531`), `m_obs` 4→5, `rank(J_UYAP)` 1→2, `rank(J_KAP)` 2, `rank(J_combined)` 3→4, singular values `[1.760, 0.975, 0.214, 0.0285, 1.0e-17]`, smallest non-zero sv `0.118→0.0285`, condition `1.7e17`, weak direction `{sigma_s:0.80, mu_s:0.59, eta:-0.07}`; every currently-modeled moment is now observed (`unavailable_moments` empty) yet status stays `NOT_IDENTIFIED` (rank 4 / dim 6) — the remaining gap is degenerate `sale_prob` + limited variation, not a missing moment. TOKİ stays `external_cross_mechanism_benchmark`; no ML / weak-supervision / fourth-source layer added.** UYAP still operator-blocked: 1 unsold auction (to break the degenerate sale probability)
335335
- [x] **UYAP outcome-taxonomy correction, invalid `uyap_sale_prob` removed, pivot to PARTIAL IDENTIFICATION + identification-aware prediction** — the actual authenticated e-Satış interface exposes four top-level states (`Satıldı`, `Birinci Alıcıya Süre Verildi`, `Malın Satışının Düşmesi`, `İhale Sonucu Girilmemiştir`); **no fifth status was invented** to unlock a sale probability. Only `Satıldı` is a terminal completed sale; settlement-pending / missing-result are **censored** (not `sold=false`), and `Malın Satışının Düşmesi` is reason-dependent (withdrawal/`Satıştan Vazgeçilmesi` is administrative, **not** a market no-trade). Because the public taxonomy cannot separate a comparable negative auction-trade class, **`uyap_sale_prob` was removed from `m_obs`, the simulated moments, and the Jacobian** (documented reason: *public UYAP outcome taxonomy does not currently identify a comparable negative auction trade class*; raw taxonomy + reason preserved for future research, never replaced with a guessed rate). The conditional moments `uyap_win_over_appraisal_mean/sd` are kept, explicitly interpreted as *winning_bid/appraised_value conditional on an observed completed sale*. **Measured effect: removing the degenerate `sale_prob=1.0` moment dropped `m_obs` 5→4 but improved conditioning — condition number `1.7e17 → 61.8` (the spurious ~0 singular value vanished), `rank(J_combined)` stays 4/dim 6.** The final inference gate pivots from forced point identification to **partial identification** `Θ_I = {θ : Q(θ) ≤ Q_min + tol}` (explicit, sensitivity-tested tolerance `tol = max(1e-4, rel·|Q_min|)`, common random numbers, reproducible sampling) via `sold structural partial` — measured `mu_b`/`sigma_b`/`auction_shift` point-like, `mu_s`/`sigma_s`/`eta` **set-identified**, with parameter trade-off correlations. `sold structural value --partial` produces an **identification-aware** closing range that separates *within-θ negotiation uncertainty* from *between-θ identification uncertainty* (`identification_status = PARTIALLY_IDENTIFIED`), never called an observed price or measured ordinary-resale accuracy. θ was **not** shrunk to recover rank; TOKİ stays external; no ML / weak-supervision / SaleProbability / fourth source added. The public-source UYAP no-trade hunt is closed
336336
- [x] **Econometric terminology correction (near-fit set) + input-conflict diagnostic; core frozen** — the near-minimum SMM criterion level set was renamed `admissible_near_fit_set` (`Θ_A`) and is explicitly **not** described as a formally estimated identified set, a confidence region, or any coverage claim: *the set of economically admissible structural parameter vectors whose SMM criterion lies within the documented near-fit tolerance of the best observed-moment fit* (the tolerance `max(1e-4, rel·|Q_min|)` is a documented numerical/sensitivity rule, **not** a sampling-calibrated cutoff). Local point-identification diagnostics are preserved (`Jacobian rank = 4`, `dim(θ) = 6`) and the reported status is **`STRUCTURALLY_UNDERIDENTIFIED`**. The prediction envelope across `Θ_A` is a **near-fit structural parameter uncertainty envelope** / *structural sensitivity range* — separating *within-θ negotiation uncertainty* from *between-θ near-fit parameter uncertainty* — never a confidence interval or a measured-coverage prediction. A `FUTURE_METHODOLOGY_NOTE` records that a formally calibrated confidence region would need an inference procedure whose criterion cutoff accounts for sampling uncertainty. Added an **input-conflict diagnostic**: `ask_to_fair_value_ratio = asking_price / fair_value` is computed and an explicit `input_conflict` warning is emitted when it falls outside documented configurable bounds (default `[0.5, 2.0]`) — the prediction is **never silently clamped or rejected**; the six economic explanations (`possible_input_error`, `geographic_anchor_mismatch`, `property_characteristic_mismatch`, `distressed_or_nonstandard_sale`, `fractional_or_encumbered_interest`, `strategic_underpricing`) are surfaced as **candidate diagnostic categories only**, never auto-assigned without evidence. Frozen: UYAP+KAP dual-mechanism core, TOKİ external benchmark, `uyap_sale_prob` excluded, the two genuine UYAP + two genuine KAP observations; no ML / weak-supervision / SaleProbability / fourth source / new mechanism. **Econometric core frozen; next is productization + dataset expansion.**
337+
- [x] **Final product surface over the frozen structural engine (honest, demo-ready)** — a polished single-page structural valuation UI (tabs **Değerle / Model Evidence / Method**) and a machine-readable API replace the legacy development form. `POST /structural/valuate` returns `methodology=structural_econometrics`, `identification_status=STRUCTURALLY_UNDERIDENTIFIED`, `coverage_claim=null`, `central_structural_estimate`, `within_theta_negotiation_interval`, `between_theta_near_fit_band`, `structural_sensitivity_range`, `ask_to_fair_value_ratio`, `input_conflict` (+ warning), genuine `2/2/5` UYAP/KAP/TOKİ counts, `jacobian_rank`/`parameter_dimension`/`near_fit_parameter_count` — and **never** a `confidence_interval` or `accuracy` field. The result hierarchy shows the central estimate, within-θ vs between-θ uncertainty, and the sensitivity envelope with the visible statement *"This is not a confidence interval and carries no frequentist coverage claim."* `GET /structural/evidence` reports the genuine public evidence honestly (UYAP conditional-on-completed-sale, KAP corporate-negotiated ≠ ordinary-resale ground truth, TOKİ external = 0 SMM moments) and **does not** display test counts as evidence; `GET /structural/method` explains the mechanism (`TCMB anchor → fair value`, `UYAP/KAP → moments`, `SMM → Θ_A`, `trade iff B≥S`, `P=η·B+(1−η)·S`, `simulation across Θ_A → structural sensitivity range`) with `η` **not** measured from KAP and appraised value **not** the auction reserve; the `[0.5, 2.0]` ask/fair conflict bounds are documented as **configurable product diagnostics, not econometric thresholds**. Input-conflict surfaces an explicit warning with the ratio and the six **candidate** explanation categories (never auto-assigned), and the estimate is **never silently clamped or rejected**. One test-backed correctness fix to the prediction summary: the identification-aware envelope no longer collapses to null when only the single best-fit θ trades zero — it is built from the near-fit configurations that actually trade, while still reporting an honest null when **no** admissible θ trades. Frozen: structural core, TOKİ external, `uyap_sale_prob` excluded, 2 UYAP + 2 KAP; no ML / weak-supervision / SaleProbability / fourth source / new mechanism / auth / billing.
337338
- [ ] **SaleProbability** model (`P(sold ≤ N days)`) trained on collected outcomes
338339
- [ ] Live, ToS-reviewed fetchers for the public label sources
339340
- [ ] Broker-vs-benchmark analytics over an aggregate anonymized dataset

0 commit comments

Comments
 (0)