Merge pull request #82 from ChBrain/fix/withheld-audit-footers

fix: withheld audit footers, Norderelbe correction, spotcheck coverage

Author: ChBrain

.githooks/pre-commit

@@ -48,7 +48,40 @@ def detect_bump_type(current: str, staged: str) -> str:
     return "NONE"
 
 
+UTF8_BOM = b"\xef\xbb\xbf"
+
+
+def strip_bom_from_staged_files() -> list[str]:
+    """Strip UTF-8 BOM from any staged text file that has one.
+
+    PowerShell Set-Content writes BOM regardless of .editorconfig. Stripping
+    at commit time keeps the repository clean without requiring each tool/shell
+    to be configured correctly.
+
+    Returns a list of file paths that were fixed.
+    """
+    staged = run_git("diff", "--cached", "--name-only")
+    fixed = []
+    for rel in staged.splitlines():
+        path = Path(rel)
+        if not path.exists() or path.suffix not in {".md", ".txt", ".py", ".sh", ".json"}:
+            continue
+        raw = path.read_bytes()
+        if raw.startswith(UTF8_BOM):
+            path.write_bytes(raw[3:])
+            subprocess.run(["git", "add", rel], check=False)
+            fixed.append(rel)
+    return fixed
+
+
 def main() -> int:
+    # Strip BOM from staged files before any other checks.
+    # Transparent fix for PowerShell Set-Content BOM injection.
+    fixed = strip_bom_from_staged_files()
+    if fixed:
+        for f in fixed:
+            print(f"  (stripped UTF-8 BOM from {f})")
+
     # Guard: prevent direct commits to main branch
     current_branch = run_git("rev-parse", "--abbrev-ref", "HEAD")
     if current_branch == "main":

.gitignore

@@ -2,3 +2,4 @@ __pycache__/
 *.py[cod]
 
 dist/
+audits/

ARCHITECTURE.md

@@ -88,7 +88,27 @@ All filenames use ASCII characters only. Underscores separate words. No hyphens
 - `piece_federal-motorway-network.md` ✗ (hyphen not allowed)
 - `place_strasse_muenchen.md` ✗ (ü/ö/ä not allowed; use ascii equivalents or spell out)
 
-The deployer (`scripts/deploy.py`) asserts uniqueness of basenames across the entire bundle and resolves all links by basename. Filename validation is enforced by `scripts/validate_general.py`.
+The deployer (`scripts/deploy.py`) asserts uniqueness of basenames across the entire bundle and resolves all links by basename.
+
+---
+
+## File Relationships
+
+```mermaid
+erDiagram
+    STATE ||--o{ ROAD : "references (Holds section)"
+    STATE ||--o{ NODE : "references (Holds section, by state border)"
+    ROAD  ||--o{ NODE : "contains (km chain in Holds)"
+    NODE  }o--|| ROAD  : "Owner: Place in road"
+    NODE  }o--|| STATE : "Owner: Place in state"
+```
+
+Key constraints:
+- A node belongs to exactly one road and exactly one state.
+- A road spans 1..n states; a state contains 1..n roads.
+- State files list only the nodes within their own borders, not all nodes on a road.
+- Folder structure is the authority: `roads/<road>/` owns a node. State membership is declared in the node's `## Owner` block and must match what the state file lists in `## Holds`.
+- Sync failures (node listed in wrong state, node missing from state Holds, state backreference in road index pointing to wrong file) are validation errors.
 
 ---
 
@@ -149,15 +169,6 @@ Subsection order follows the road's direction from its starting terminus (first
 * [km 348.0]: [Neubrandenburg-Ost](place_a20_39_neubrandenburg_ost.md)
 ```
 
-**Validation (STRICT):** `scripts/validate_roads.py` enforces:
-- Road index files have `Owner: - Project: Autobahn` only (no other items)
-- Multi-state roads have subsections by state
-- Each subsection has state name header and backreference
-- State backreferences are valid (files exist)
-- Kilometre sequences within states are strictly ascending
-- No cross-state entry mixing (all entries for one state must be in its subsection)
-- Entry format matches: `* [km X.X]: [Name](place_file.md)`
-
 ---
 
 ### Exit
@@ -330,10 +341,27 @@ The single author-facing rule: **every file basename in the world is unique**.
 
 ---
 
+## States
+
+The 16 federal state files live in `states/place_<state>.md`. Each state file is a Place that aggregates roads and their nodes.
+
+**Owner** carries only `- Project: Autobahn`.
+
+**Shown** is the state as experienced from the motorway network - landscape, approach, what the road reveals about the territory.
+
+**Holds** is the road list. For each road the state contains, there is a subsection headed by the road link. Under each road, the state lists only the nodes within its own borders in kilometre order. Nodes in other states on the same road are not listed.
+
+**Offers** is what the state's network makes available - connections, destinations, facts about density and coverage.
+
+**Withheld** is what the motorway passes through but does not show - geography, cities, landmarks unreachable without leaving the road.
+
+**Naming:** `states/place_<bundesland>.md` using the full lowercase German state name in ASCII (e.g. `place_schleswig_holstein.md`, `place_mecklenburg_vorpommern.md`).
+
+---
+
 ## To document
 
 - **Geographic coordinates** - each place node to carry latitude, longitude, and altitude (from OSM or BASt sources). Enables geographic validation: compass directions verified against bearing, distances against geodetic calculations, elevation changes detected. Lookup table vs. per-file embedding still open. Proposed: validate directions against bearing; validate distances against haversine; detect anomalies (altitude spikes, direction contradictions). Requires authoritative source identification and accuracy tolerance definition.
-- **State files** - the 16 federal state files in `states/`, their structure, subtitle tagline pattern, and how they aggregate roads and nodes. Needs its own `## State` chapter and `validate_states.py` script.
 - **Naming conventions in use** - CONVERTED: German infixes replaced with English (`_bruecke_` → `_bridge_`, `_raststaette_` → `_service_`). Road-tied stops (service/rest/roadhouse) now require road prefix: `place_{road}_service_*`, `place_{road}_rest_*`, `place_{road}_roadhouse_*`; optional exit numbers if official AS-number exists. ✅ Migration complete. Tunnel naming still pending.
 - **Navigation links placement** - FIXED: neighbour links (distance + direction markers) moved from `## Shown` to `## Holds` across 51 files (A1, A7, A20, A23, A24, A25, A210, A215). Validator `validate_navigation_links.py` enforces rule; all 187 files pass.
 - **Engine files** - `instructions.md`, `stack.md`, and the pieces in `engine/` (kilometerstein, driver seat, infotainment, etc.)

README.md

@@ -75,7 +75,7 @@ The world holds it. It does not resolve.
 
 ## How this is built
 
-Authored by Kai Schlüter (ChBrain) with AI-assisted drafting. The per-file workflow is: a model proposes several drafts, the facts are checked against authoritative sources, then the final voice is rewritten by hand. The aim is to use facts (not copyrightable) carried in the author's own expression.
+Authored by Kai Schlüter (ChBrain) with AI-assisted drafting. The per-file workflow is: a model proposes several drafts, the facts are checked against authoritative sources, then the final voice is rewritten. The aim is to use facts (not copyrightable) carried in the author's own expression.
 
 This is not an academic paper - the source list in [REFERENCES.md](REFERENCES.md) names the sources used most often, not all of them. If you spot a passage that closely paraphrases a specific source, or a factual error, please open an issue and link the source.
 

REFERENCES.md

@@ -10,7 +10,7 @@ Authored by Kai Schlüter (ChBrain) with AI-assisted drafting. The per-file work
 
 The intent is to use facts (which are not copyrightable) carried in the author's own expression. Occasional close-paraphrase or factual drift may slip through; if you spot it, please open an issue and link the source.
 
-`scripts/spotcheck_withheld.py` is a sampling helper for periodic review: it picks N random in-scope Withheld blocks (deterministic per seed) and prints them as a markdown brief with audit instructions. The audit itself is performed by whichever LLM session runs the script - no API key required.
+`scripts/validate_withheld.py` is a sampling helper for periodic review: it picks N random in-scope Withheld blocks (deterministic per seed) and prints them as a markdown brief with audit instructions. The audit itself is performed by whichever LLM session runs the script - no API key required.
 
 ---