Skip to content

Commit c478335

Browse files
committed
docs: document UYAP Evidence Ingestion Pipeline V1
README: add a UYAP Evidence Ingestion Pipeline V1 section (purpose as a provenance-aware data-acquisition subsystem; browser-assisted not authentication-bypassing; user-controlled session; manual artifact-import fallback; discovery->audit->review->explicit admission workflow; explicit Ihale Bedeli semantics; terminal completed-sale requirement; human review; privacy; honest live-access limitation), plus the ingestion package in Project Structure and the 'sold uyap' CLI. DEVELOPMENT_HISTORY: add the ingestion V1 milestone. Documentation only.
1 parent a476c18 commit c478335

2 files changed

Lines changed: 49 additions & 0 deletions

File tree

README.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ No fabricated data is ever served, and the system never reports a measured ordin
2727
- [How It Works](#how-it-works)
2828
- [Structural inference engine](#structural-inference-engine)
2929
- [Provenance-Audited Public Structural Evidence](#provenance-audited-public-structural-evidence)
30+
- [UYAP Evidence Ingestion Pipeline V1](#uyap-evidence-ingestion-pipeline-v1)
3031
- [Optional direct-label validation channel](#optional-direct-label-validation-channel)
3132
- [Data Sources](#data-sources)
3233
- [Install](#install)
@@ -209,6 +210,30 @@ The **SMM moment vector uses exactly four moments** (two UYAP, two KAP). TOKİ a
209210
- The consumer direct-label channel is **frozen** and never contributes to SMM.
210211
- `sold structural dataset` reports the genuine audited counts per source **separately** from fixtures, demo data, and near-fit search candidates.
211212

213+
## UYAP Evidence Ingestion Pipeline V1
214+
215+
A **provenance-aware data-acquisition subsystem** ([`src/sold/ingestion/uyap/`](src/sold/ingestion/uyap/)) that reduces the manual work of discovering, collecting, extracting, auditing, reviewing, and admitting UYAP e-Satış completed-sale evidence. It is **not** a new methodology or pricing mechanism and does **not** modify the frozen structural core; admission writes to the existing genuine UYAP schema and preserves the UYAP P/Q moment definition.
216+
217+
**Workflow.** `discovery → collection → extraction → same-asset reconciliation → rule-based completed-sale audit → human review → explicit admission → existing UYAP evidence schema`. Extraction is deterministic (no ML, no weak supervision, no classifier); each field is traceable to the artifact it came from. **Audit is not admission** — a parser never writes genuine `uyap.json` directly; admission is a separate, explicit, idempotent operator action that validates the existing schema and dedupes by `public_record_id`.
218+
219+
**Completed-sale admission rule** (formalizing the seven genuine observations + Batch 1): a candidate is admissible only with (A) an auditable appraisal `Q` for the same asset, (B) an **explicit official İhale Bedeli**, and (C) terminal completed-sale evidence (`Satıldı` / `Satış İşlemleri Tamamlandı`). The **auction-price numerator is always the explicit İhale Bedeli** — never the *Ödenmesi Gereken Bedel*, deposit-adjusted balance, ownership-share settlement, creditor-setoff, or KDV-adjusted amount; KDV never adjusts `P` or `Q`; `ALACAĞA MAHSUBEN` never invalidates an explicit İhale Bedeli. Non-terminal records (e.g. `Birinci Alıcıya Süre Verildi`) become `EXCLUDED_NON_TERMINAL` in [`validation/structural/uyap_candidates.json`](validation/structural/uyap_candidates.json) — never admitted, never a negative sale-probability observation (`uyap_sale_prob` is never created). Ambiguous candidates (missing appraisal / missing explicit İhale Bedeli / missing terminal evidence / reconciliation ambiguity) are surfaced to a **human-review queue** with the exact blocking reason and are never silently promoted.
220+
221+
**Browser-assisted, not authentication-bypassing.** The optional browser collector (Playwright, `pip install -e ".[browser]"`) operates **only within a user-controlled, already-authenticated or public session** (attach to a browser you launched via a CDP endpoint, or a local profile you signed into yourself). It **never** automates e-Devlet login, MFA or CAPTCHA, never bypasses access controls, and never stores credentials, cookies, session tokens, or browser profiles in the repository (raw artifacts and profiles live under gitignored `data/`). If a live browser is unavailable, the **manual artifact-import** path (saved HTML/PDF/text) is the fallback.
222+
223+
**Privacy.** Only non-personal institution / official file identifier / property / economic / public-record fields are retained. Party, debtor, creditor, counsel, and personal identifiers (names, TC IDs, phones, IBANs, accounts) are never propagated into normalized extraction or analytical records.
224+
225+
```bash
226+
sold uyap discover --institution "Ankara ... Satış Memurluğu" --file-id "2026/43 Satış"
227+
sold uyap import-artifacts --candidate-id <id> --type auction_result --path saved_result.html
228+
sold uyap extract --candidate-id <id> # deterministic fields (not admission)
229+
sold uyap audit --candidate-id <id> # rule-based completed-sale audit (not admission)
230+
sold uyap review # human-review queue with blocking reasons
231+
sold uyap admit --candidate-id <id> # EXPLICIT, idempotent admission to uyap.json
232+
sold uyap status # discovered / audited / admissible / admitted
233+
```
234+
235+
> **Live UYAP access status.** Live end-to-end UYAP access was **not** available in the development environment and was **not** tested; there is **no** official UYAP API integration. The browser adapter is implemented and its prerequisites documented honestly, the deterministic parsers run against local fixtures and manually saved artifacts, and the **automated test suite requires no network**.
236+
212237
## Optional direct-label validation channel
213238

214239
Separately from the structural evidence above, `sold` retains a **frozen** listing-outcome channel that lets a broker or seller record the *outcome* of a listing and receive simple **non-ML negotiation analytics** in return. This channel is **not** part of the structural model:
@@ -351,6 +376,7 @@ src/sold/
351376
flywheel/ # frozen optional direct-label validation channel
352377
consumer/ # frozen consumer submission path + quality gate
353378
scraper/ # ToS-respectful local-example pipeline (no live scraping)
379+
ingestion/uyap/ # UYAP evidence ingestion V1 (discovery→audit→explicit admission)
354380
tuik/ # TÜİK client
355381
db/ # SQLAlchemy models + schema
356382
cli.py # `sold` command-line interface (incl. `sold structural ...`)

docs/DEVELOPMENT_HISTORY.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,3 +161,26 @@ milestones:
161161
conditioning improved; the numerical near-fit search — a diagnostic **separate from and
162162
not establishing** identification — re-measured to `STABLE` at the audited budgets on the
163163
better-fitting 7-record evidence. Full suite: **217 passed** (215 baseline + 2 Batch-1 tests).
164+
165+
- **UYAP Evidence Ingestion Pipeline V1 (data-acquisition subsystem)** — added
166+
[`src/sold/ingestion/uyap/`](../src/sold/ingestion/uyap/): a provenance-aware
167+
`discovery → collection → extraction → same-asset reconciliation → rule-based
168+
completed-sale audit → human review → explicit admission` pipeline plus a `sold uyap`
169+
operator CLI (`discover` / `import-artifacts` / `collect` / `extract` / `audit` /
170+
`review` / `admit` / `status`). It **does not touch the frozen structural core**:
171+
admission validates against the existing `normalize_auction` schema, writes genuine
172+
`uyap.json` idempotently (dedupe by `public_record_id`), and preserves the UYAP P/Q
173+
moment (numerator = explicit **İhale Bedeli**, never *Ödenmesi Gereken Bedel* / deposit /
174+
share / setoff / KDV; KDV never adjusts P or Q; `ALACAĞA MAHSUBEN` never invalidates an
175+
explicit İhale Bedeli). Non-terminal records → `EXCLUDED_NON_TERMINAL` (never a negative
176+
sale-probability observation; `uyap_sale_prob` never created). Extraction is deterministic
177+
(no ML / weak supervision / classifier). The browser collector (optional Playwright extra)
178+
operates **only** within a user-controlled, already-authenticated or public session —
179+
never automating e-Devlet login / MFA / CAPTCHA and never committing credentials, cookies,
180+
tokens, or browser profiles; a manual artifact-import fallback keeps the pipeline usable
181+
offline. **Live UYAP access was not available/tested in the dev environment and there is no
182+
official UYAP API integration**; deterministic parsers run against local fixtures and the
183+
automated suite needs no network. Added 20 offline regression/behavioral tests (six audited
184+
price-semantics cases + admission idempotency/duplication/freeze). Full suite: **237 passed**
185+
(217 baseline + 20 ingestion tests). Structural moments, `θ`, mechanism, SMM, `Θ_A`,
186+
`conditional_on_trade`, and the numerical-search convention are unchanged.

0 commit comments

Comments
 (0)