Skip to content

Latest commit

 

History

History
528 lines (340 loc) · 12.3 KB

File metadata and controls

528 lines (340 loc) · 12.3 KB

Urban Dossier v3.7.6 Engineering

1. Why v3.7.6 Exists

v3.7.6 is the first version explicitly aligned to the imported demo frontend in interactive-map-explorer while preserving the backend architecture we settled on in v3.6 and v3.7:

  • backend remains the source of truth for analysis
  • overview remains a navigation layer, not a report generator
  • detail remains the place where deterministic ranking happens
  • final report remains a separate confirm action
  • frontend should change as little as possible

This version therefore optimizes for:

  1. minimal frontend breakage
  2. backend-first implementation
  3. stable contracts that can keep evolving after the frontend catches up
  4. a Pensar-friendly architecture with thin adapters instead of duplicated logic

2. Final Product Interpretation

2.1 Overview

Overview is the city-scale map experience.

It supports:

  • general
  • amenities
  • transit
  • safety

Overview does not generate reports.

Overview should help the user answer:

  • where should I click next?
  • which part of the city looks elevated under this lens?

2.2 Detail Preview

Detail preview begins after the user clicks a point on the map.

The user can then adjust:

  • radius: 200m, 500m, 1000m
  • priority ranking: Amenities, Transit, Safety

Preview is deterministic and does not call the LLM.

Preview should answer:

  • what rises to the top around this point right now?
  • how does that change if I change the radius or reorder the priorities?

2.3 Detail Final

The detail panel contains a dedicated Generate Report button.

Only after the user clicks that button do we generate the final local governance brief.

The final report:

  • reuses the same deterministic detail pipeline
  • adds one LLM pass at the end
  • stays grounded in evidence and local context

3. Final Interaction Flow

Open app
  ->
Overview map loads on "general"
  ->
User optionally switches map lens (general / amenities / transit / safety)
  ->
User clicks a point
  ->
Detail preview request runs
  ->
User changes radius and/or priority order
  ->
Detail preview refreshes
  ->
User clicks "Generate Report"
  ->
Final detail request runs
  ->
LLM writes local governance brief

4. Interface Topology

v3.7.6 keeps three layers.

4.1 Browser / React

The React frontend remains the user-facing shell.

It should continue to:

  • render the UI
  • host drag ordering and radius controls
  • own the detail panel state

It should not:

  • reimplement analysis logic
  • invent synthetic scores

4.2 Node Map Server

server.js remains in the stack, but only as a thin integration layer.

Responsibilities:

  • serve local vector tiles and fonts
  • serve the built frontend if needed
  • proxy a small whitelist of API routes to Python
  • keep legacy demo endpoints alive as compatibility aliases

It should not:

  • perform real analysis
  • fabricate overview/detail results
  • diverge from backend truth

4.3 Python Backend

Python remains the analysis system of record.

Responsibilities:

  • overview contract
  • detail preview
  • detail final
  • deterministic signals / trends / priorities / evidence
  • final brief generation

5. Canonical Naming

Frontend-facing category labels stay aligned to the imported demo:

  • Amenities
  • Transit
  • Safety

Backend canonical ids in v3.7.6:

  • amenities
  • transit
  • safety
  • building
  • overall

Accepted aliases:

  • general -> overall
  • overall -> overall
  • facilities -> amenities
  • traffic -> transit

6. API Strategy

6.1 Canonical Python APIs

  • GET /api/health
  • GET /api/categories
  • GET /api/coverage
  • POST /api/overview
  • POST /api/detail/preview
  • POST /api/analyze-point
  • POST /api/watchlist/run

6.2 Thin Node Compatibility APIs

These exist to preserve the current frontend shape with minimal rewiring:

  • GET /api/render/global
  • POST /api/render/local

Behavior:

  • render/global adapts to backend overview
  • render/local adapts to backend detail preview

Node may also proxy through:

  • GET /api/health
  • GET /api/categories
  • GET /api/coverage
  • POST /api/detail/preview
  • POST /api/analyze-point
  • POST /api/watchlist/run

7. Final Decision on Frontend Wiring

We do not force the frontend to abandon the demo map APIs immediately.

Instead:

  1. Map.tsx keeps using /api/render/global and /api/render/local
  2. App.tsx starts using /api/detail/preview and /api/analyze-point
  3. Node translates the legacy render endpoints into backend calls

This is the least disruptive path and best matches the current frontend code.

8. Radius Semantics

Allowed detail radii:

  • 200m
  • 500m
  • 1000m

Recommended interpretation:

  • 200m: immediate frontage / near-building context
  • 500m: walkable local context
  • 1000m: broader neighborhood context

The report must explicitly mention the selected radius.

9. Signal-to-Radius Rules

9.1 Strictly follow selected radius

These should use the selected user radius directly:

  • collisions
  • rodent inspections
  • 311 complaints
  • restaurant inspections
  • LinkNYC
  • public toilets
  • street trees
  • other point/facility signals

9.2 Building-specific radius

Building signals must use a bounded near-building radius:

building_radius = min(selected_radius, 250m)

This prevents building-specific risk from being washed out at 1000m.

9.3 ZIP / area context does not follow radius

These stay area-context proxies:

  • EMS
  • Fire

They must be described as ZIP-level context in the final report.

9.4 Polygon / large-area proxies

First implementation may continue to use simplified access/proxy logic.

If polygon precision is limited, the report should acknowledge it in data_gaps.

10. Severity and Baseline Normalization

Final intended approach:

  • radius-specific baselines for 200m, 500m, 1000m

Interim fallback:

  • density-style normalization when radius-specific baselines are not ready

This prevents 1000m from looking worse purely because counts are larger.

11. Overview Output Contract

Overview should return enough structure for the frontend to color cells later, even if the current map renderer still adapts them into point-like hints.

Recommended cell fields:

  • cell_id or h3
  • latitude
  • longitude
  • overall_score
  • category_scores
  • building_stress_score
  • risk_level

11.1 General / overall layer

general on the frontend corresponds to backend overall.

overall is produced from fixed platform weights, not user drag order.

11.2 Overview fallback

If overview layers are not ready:

  • return overview_ready: false
  • return empty cells
  • return a UI message prompting point-based detail analysis

This must not crash the frontend.

12. Detail Preview Contract

Preview is the main deterministic local analysis payload.

It should include:

  • target
  • priority_profile
  • priority_actions
  • why_now
  • current_state
  • detail_items
  • evidence_table
  • data_gaps
  • scores

It should not include:

  • LLM-generated longform brief

13. Detail Final Contract

Final detail is preview plus:

  • report_summary
  • report_markdown

The final report must be generated only when the user explicitly confirms with Generate Report.

14. Building Strategy

We keep 4A as the final design.

14.1 Overview

Overview may expose aggregated building stress fields, but building is not a primary toggle category in the current map UX.

14.2 Detail

Detail includes a separate building section.

That section appears only when building findings exist.

Recommended outputs:

  • building_stress_score
  • building_flags
  • open_class_c_count
  • aep_flag
  • severity_level

This keeps the backend ready for future building-specific map styling without overcommitting the frontend today.

15. Detail Panel Final Behavior

15.1 On click

When the user clicks a point:

  • open detail panel
  • run preview
  • show local scores, top priorities, evidence, and preview narrative

15.2 While changing radius / order

When the user changes:

  • radius
  • drag order

the app refreshes preview only.

15.3 On Generate Report

When the user clicks Generate Report:

  • keep the same selected point
  • keep the same radius
  • keep the same priority order
  • call final analysis endpoint
  • replace preview narrative with final report text

16. Final Report Style

The report should read like a concise local governance brief for managers, operators, or community-facing decision-makers.

It should not read like:

  • a dashboard dump
  • a rigid form
  • a generic area summary

16.1 Hard requirements

The prompt must ensure the report:

  • states concrete conclusions
  • mentions the selected radius
  • distinguishes local-radius findings from ZIP-level service context
  • explains why the current priorities rose to the top
  • acknowledges data gaps

16.2 Soft freedom for the model

The prompt should not force identical paragraphing every time.

The model should be free to adapt to:

  • good evidence
  • bad evidence
  • mixed evidence
  • sparse evidence

16.3 Direction

The target is not a summary.

The target is:

a specific, evidence-grounded local conclusion about what matters most here right now

17. Overview Context in Final Report

Final reports may include a light overview_context when available.

This should remain lightweight.

Example use:

  • “This point sits within a citywide-elevated safety overview cell...”

It should not become a separate dominant report section.

If overview context is unavailable, the report should still complete successfully.

18. What Changed in v3.7.6 Code

18.1 Backend

  • schema version moved to v3.7.6
  • canonical categories align to frontend naming
  • detail preview remains a first-class endpoint

18.2 Node

  • /api/render/global now adapts from backend /api/overview
  • /api/render/local now adapts from backend /api/detail/preview
  • Node proxies a safe whitelist of backend routes
  • Node no longer needs synthetic analysis logic to serve the current frontend

18.3 Frontend

  • Vite proxies /api to the Node server
  • map keeps existing render endpoints
  • detail panel uses real preview/final requests
  • detail panel gets a dedicated Generate Report button
  • building section appears only when building findings exist

19. Current Known Frontend Limitations

Even after v3.7.6, some limitations remain.

19.1 Search

The current search is still a lightweight offline landmark/coordinate resolver in the frontend.

It is acceptable for now because it only helps place the user on the map.

19.2 Overview visuals

The existing map visual language still approximates overview coloring by adapting backend overview cells into point-based building coloring.

That is acceptable for this stage because:

  • it preserves the existing frontend
  • it keeps the backend contract stable
  • it can later be upgraded to native cell or polygon rendering

19.3 Right panel layout

The current detail panel is still visually score-heavy.

However, its data flow is now moving toward:

  • preview priorities first
  • report on explicit confirm
  • evidence-backed narrative

20. Pensar-Friendly Design

v3.7.6 keeps the integration strategy compatible with the “Least Likely to get Hacked” direction:

  • Python remains canonical
  • Node remains a thin proxy, not a second analysis engine
  • category ids are normalized
  • radius is allow-listed
  • render endpoints are compatibility wrappers, not free-form arbitrary queries

Future hardening should continue to tighten:

  • CORS
  • token-based demo auth
  • request budgets
  • watchlist size
  • backend timeout behavior

21. Acceptance Criteria

v3.7.6 is considered landed when:

  1. overview buttons still work without rewriting the map component
  2. clicking a point opens detail preview from the real backend
  3. changing radius/order refreshes preview
  4. clicking Generate Report produces final report text
  5. building section appears only when building findings exist
  6. frontend no longer fabricates detail scores and summaries locally
  7. Node does not invent synthetic analysis responses for active product paths

22. Recommended Next Steps After v3.7.6

  1. harden Node and Python for Pensar
  2. add overview_context into final detail payload
  3. improve real overview layer availability
  4. replace overview point adaptation with true cell rendering when frontend is ready
  5. promote batch/watchlist only after detail flow is fully stable