blocksense-v3.7.5-engineering.md

Urban Dossier NYC v3.7.5 - Backend / Frontend Interface Alignment Engineering Document

Target reader: backend engineer / frontend engineer / Codex / Claude Code / team manager

Date: 2026-04-11 Scope: full backend contract aligned to the current demo frontend, the v3.6 backend architecture, and the v3.7 open-question decisions This is the new working engineering source for interface alignment.

---

0. Why v3.7.5 Exists

v3.6 established the backend architecture:

• overview mode
• detail mode
• watchlist mode
• direct provider vs skill provider
• deterministic trend / priority / evidence / report pipeline

v3.7 then captured the decisions still being settled:

• report trigger timing
• radius logic
• building handling
• prompt style direction
• overview vs detail boundary

At the same time, the imported demo frontend introduced a very concrete implementation reality:

• it already ships a working offline map resource path
• it already expects general / safety / transit / amenities
• it already has a local map render API shape
• it does not yet speak the current backend contract directly

Therefore, v3.7.5 exists to answer one specific question:

How do we make the backend line up with the current frontend demo without losing the stronger architecture we already agreed on?

This document is the answer.

---

1. Product Interpretation For v3.7.5

1.1 User-visible states

The product has three backend-relevant states:

1. Overview
   • whole-map view
   • no report generation
   • fixed map buttons only
2. Detail Preview
   • after a map click
   • user chooses radius and ranking
   • backend recomputes local structured results
   • no LLM report yet
3. Detail Final
   • user presses a confirm / generate button
   • backend reuses the same local computation and adds the final report

1.2 What overview does

Overview helps the user locate potentially important areas.

Overview should support:

• one general / overall map
• one single-category map per category button

Overview should not:

• run LLM report generation
• apply user drag order
• produce custom weighted permutations of citywide maps

1.3 What detail does

Detail is where the real local analysis happens.

Once the user clicks a point:

• the backend uses the selected point
• the backend uses the selected radius
• the backend uses the user priority order
• the backend recomputes local issues in realtime
• the backend returns local structure
• on confirm, the backend additionally returns the final report

1.4 What watchlist does

Watchlist is still part of the architecture, but it is not the primary frontend path right now.

For v3.7.5:

• keep watchlist in the backend contract
• keep it structured-first
• do not let it drive the main frontend interaction design

---

2. Ground Truth We Must Design Around

2.1 Frontend truth

The imported demo frontend currently uses:

• a local Node/Express server in server.js
• a local MBTiles file in osm-2020-02-10-v3.11_new-york_new-york.mbtiles
• frontend category labels:
  • Amenities
  • Transit
  • Safety
• frontend map tags:
  • general
  • amenities
  • transit
  • safety

2.2 Backend truth

The Python backend already has:

• deterministic point analysis
• trend engine
• priority engine
• evidence generation and verification
• overview fallback
• direct vs skill provider abstraction

But before this revision, the backend and frontend still disagreed on:

• category naming
• preview vs final report separation
• API route shapes
• data types expected by the frontend

2.3 Security / Pensar truth

We want the system to remain compatible with a future Pensar/Apex hardening pass.

Therefore:

• we should avoid multiplying business logic across duplicate endpoints
• we should prefer a canonical backend contract
• any compatibility layer should be thin
• the browser should not need to learn many separate analysis endpoints if the local frontend server can proxy

That matters directly for v3.7.5.

---

3. Canonical Category Naming

3.1 Final category IDs for v3.7.5

From this version onward, the canonical backend category IDs should match the frontend language:

• amenities
• transit
• safety
• building

building remains:

• available in detail
• not one of the user-drag main categories
• not a primary overview button

3.2 Preserved aliases for compatibility

The backend should still accept these aliases:

• facilities -> amenities
• traffic -> transit
• general -> overall
• overall -> overall

This is important for:

• older docs
• older code
• teammate experiments
• compatibility during transition

3.3 Category meaning

Safety

• rodent
• sanitation 311
• EMS ZIP proxy
• Fire ZIP proxy

Transit

• collisions
• mobility-related local incident pressure

Amenities

• parks access proxy
• public toilets
• LinkNYC
• restaurant / local facility context

Building

• housing violations
• AEP
• future building-specific stress signals

---

4. Overview / Preview / Final State Machine

4.1 Overview state

User interaction:

1. open map
2. choose one of:
   • general
   • safety
   • transit
   • amenities

Backend behavior:

• returns overview cells for the chosen map layer
• does not generate a report
• does not use drag order

4.2 Detail preview state

User interaction:

1. click point
2. choose radius
3. drag category order
4. preview local structure

Backend behavior:

• recomputes local deterministic analysis
• returns:
  • current_state
  • detail_items
  • priority_actions
  • why_now
  • evidence_table
  • data_gaps
  • scores
• does not call the LLM

4.3 Detail final state

User interaction:

1. click confirm / generate report

Backend behavior:

• reuses the same local deterministic analysis
• then generates the final report

This separation is now explicit and intentional.

---

5. API Topology

5.1 Canonical backend API

The Python backend should own the canonical analysis contract:

• GET /api/health
• GET /api/categories
• GET /api/coverage
• POST /api/overview
• POST /api/detail/preview
• POST /api/analyze-point
• POST /api/watchlist/run

5.2 Frontend compatibility layer

The frontend demo currently expects:

• GET /api/render/global
• POST /api/render/local

These should not become independent business APIs in the backend.

Instead, v3.7.5 recommends:

• keep the canonical analysis API listed above
• let the local demo server provide a thin adapter / proxy if needed

Recommended mapping:

• GET /api/render/global?tag=general|safety|transit|amenities -> translate to POST /api/overview
• POST /api/render/local -> translate to POST /api/detail/preview

5.3 Why this is the right compromise

This gives us:

• backend-first architecture
• frontend compatibility
• less duplicated logic
• easier Pensar review later
• cleaner migration path once the frontend fully speaks canonical endpoints

---

6. API Contracts

6.1 GET /api/categories

Purpose:

• tell the frontend which category IDs and labels exist
• provide default order
• provide alias information

Example response:

{
  "schema_version": "v3.7.5",
  "default_order": ["amenities", "transit", "safety"],
  "overview_tags": ["general", "safety", "transit", "amenities"],
  "aliases": {
    "general": "overall",
    "overall": "overall",
    "facilities": "amenities",
    "traffic": "transit"
  },
  "categories": [
    {
      "category_id": "amenities",
      "label": "Amenities",
      "map_driving": true,
      "detail_rankable": true,
      "signals": ["parks_access", "public_toilets", "linknyc", "restaurant_context"]
    }
  ]
}

6.2 POST /api/overview

Purpose:

• return the selected overview layer

Request:

{
  "view_mode": "overall",
  "category_id": null,
  "viewport": {
    "north": 40.86,
    "south": 40.68,
    "east": -73.90,
    "west": -74.05
  },
  "zoom": 11,
  "render_mode": "h3_cells"
}

Category request variant:

{
  "view_mode": "category",
  "category_id": "transit",
  "viewport": {
    "north": 40.86,
    "south": 40.68,
    "east": -73.90,
    "west": -74.05
  },
  "zoom": 11,
  "render_mode": "h3_cells"
}

Response shape:

{
  "schema_version": "v3.7.5",
  "mode": "overview",
  "view_mode": "category",
  "category_id": "transit",
  "layer_mode": "h3_r8",
  "cells": [],
  "coverage": {
    "overview_ready": true,
    "available_categories": ["overall", "safety", "transit", "amenities"],
    "missing_categories": []
  },
  "resolved_data_mode": "direct"
}

6.3 POST /api/detail/preview

Purpose:

• compute full local deterministic analysis
• do not generate final report

Request:

{
  "latitude": 40.7580,
  "longitude": -73.9855,
  "radius_m": 500,
  "priority_order": ["amenities", "transit", "safety"],
  "time_window_days": 365
}

Important:

• radius_m is restricted to:
  • 200
  • 500
  • 1000
• priority_order accepts aliases but is normalized to canonical category IDs

Response shape:

{
  "schema_version": "v3.7.5",
  "mode": "detail_preview",
  "target": {},
  "priority_profile": {
    "order": ["amenities", "transit", "safety"],
    "weights": {
      "amenities": 1.0,
      "transit": 0.72,
      "safety": 0.52
    }
  },
  "priority_actions": [],
  "why_now": [],
  "current_state": {},
  "detail_items": {},
  "trends": {},
  "patterns": [],
  "evidence_table": [],
  "data_gaps": [],
  "scores": {},
  "preview_ready": true
}

6.4 POST /api/analyze-point

Purpose:

• run the same detail computation as preview
• then generate the final report

Request:

{
  "latitude": 40.7580,
  "longitude": -73.9855,
  "radius_m": 500,
  "priority_order": ["amenities", "transit", "safety"],
  "time_window_days": 365,
  "include_report": true
}

Response shape:

{
  "schema_version": "v3.7.5",
  "mode": "detail",
  "target": {},
  "priority_profile": {},
  "priority_actions": [],
  "why_now": [],
  "current_state": {},
  "detail_items": {},
  "trends": {},
  "patterns": [],
  "evidence_table": [],
  "data_gaps": [],
  "scores": {},
  "report_summary": "",
  "report_markdown": ""
}

6.5 POST /api/watchlist/run

Purpose:

• batch reuse of detail logic

For v3.7.5, this remains backend-facing and secondary.

---

7. Map Resource Ownership

7.1 What the frontend demo already owns

The current demo already owns:

• MBTiles storage
• vector tile serving
• glyph serving
• local static map resource delivery

Therefore, the Python backend should not take ownership of:

• /tiles/{z}/{x}/{y}.pbf
• font PBFs
• map style resources

7.2 What the analysis backend owns

The Python backend owns:

• overview analysis payloads
• detail preview payloads
• detail final report payloads
• categories / coverage / health metadata

This separation is important because:

• it keeps the analysis backend focused
• it reduces accidental duplication
• it makes Pensar review cleaner

---

8. Radius Logic

8.1 Allowed radii

The only user-facing radii for v3.7.5 are:

• 200m
• 500m
• 1000m

The backend should reject arbitrary radii.

8.2 Signals that strictly follow user radius

These signals should use the selected radius directly:

• collisions
• rodent inspections
• 311 complaints
• restaurant inspections
• LinkNYC
• public toilets
• street trees
• other point/facility context datasets

8.3 Building radius

Building-specific signals should use:

building_radius = min(selected_radius, 250m)

This means:

• selected 200m -> building radius 200m
• selected 500m -> building radius 250m
• selected 1000m -> building radius 250m

8.4 ZIP / area proxies

These do not follow the selected radius:

• EMS
• Fire
• future ZIP/CD service proxies

They should remain:

• area-level context
• not point-radius claims

8.5 Parks / large polygon rule

For the first implementation:

• use nearby access / local proxy logic
• allow acreage to remain approximate
• state that approximation explicitly in data_gaps

8.6 Radius normalization

Changing radius changes raw counts.

Therefore:

• raw count alone must not drive severity

Preferred long-term direction:

• radius-specific baselines for:
  • 200m
  • 500m
  • 1000m

Allowed interim fallback:

• density / area-normalized scaling

---

9. Detail Computation Logic

9.1 Deterministic first

Preview and final report should both use the same deterministic core:

1. resolve local target context
2. query local signals
3. query local historical windows
4. compute trends
5. detect patterns
6. compute priority actions
7. build evidence
8. compute scores

The final report call simply adds:

9. generate markdown brief

9.2 Why preview and final must share a core

If preview and final use different logic:

• the UI will drift
• the report will not match the preview
• trust will degrade

Therefore:

• preview is not a toy
• final is not a second analysis pass with different rules

They must share the same computed state.

---

10. Priority Weighting

10.1 User ranking is detail-only

User ranking should affect:

• preview priority ordering
• final report emphasis

User ranking should not affect:

• overview layer generation

10.2 Weight rule

Use exponential decay:

weight = decay ^ (rank - 1)

With:

• decay = 0.72

10.3 Example

If user order is:

1. amenities
2. transit
3. safety

Then weights are approximately:

• amenities = 1.00
• transit = 0.72
• safety = 0.52

10.4 Alias normalization

Frontend may still send:

• Amenities
• Transit
• Safety

or older IDs like:

• facilities
• traffic

Backend should normalize them before ranking.

---

11. Building Handling (Confirmed Choice 4A)

11.1 What 4A means in practice

Building signals should be a dedicated detail section, not a main draggable category.

That means:

• building is computed in detail mode
• building is visible in detail mode
• building does not have to appear as a top-level overview button

11.2 Why this is the right choice

It avoids mixing two different questions:

• “Is this area under local pressure?”
• “Is there a nearby building-specific issue?”

Those are related, but not identical.

11.3 How building should appear in the API

In current_state:

• building module summary

In detail_items:

• building_flags

Future-ready output examples:

{
  "building_stress_score": 31,
  "severity_level": "high",
  "open_class_c_count": 3,
  "aep_flag": true
}

11.4 Report behavior

Per the decision:

• building section should appear only if there are actual building findings

No empty building section should be forced.

---

12. Detail Report Logic

12.1 Report trigger

The final report should only be generated after:

1. point selected
2. radius selected
3. priority order selected
4. confirm button pressed

12.2 Required content

The report must explicitly include:

• selected point
• selected radius
• borough if available
• ZIP if available
• active priority order

12.3 Direction of the report

The report must aim for:

• concrete conclusions
• concrete local priorities
• evidence-backed interpretation

The report must avoid:

• bland summaries
• metric dumping
• pretending every result is alarming

12.4 Prompt philosophy

The prompt should be:

• strict about direction
• flexible about surface writing

Meaning:

• the model must write real conclusions
• the model may decide how to phrase them
• the model must faithfully reflect whether the result is:
  • good
  • bad
  • mixed
  • sparse

12.5 Radius language requirement

The prompt must instruct the model to:

• explicitly mention the selected radius
• distinguish local radius-based findings from ZIP-based context

This is essential for correctness.

---

13. Overview Context In The Detail Report

13.1 Current status

This is optional, not required for first run.

13.2 Recommended future enhancement

Add a light overview_context object to the detail payload:

{
  "overview_context": {
    "overall_cell_level": "elevated",
    "category_cells": {
      "safety": { "level": "high", "percentile_city": 88 },
      "transit": { "level": "medium", "percentile_city": 61 },
      "amenities": { "level": "low", "percentile_city": 35 }
    }
  }
}

13.3 How the report may use it

The report may add a single light sentence like:

• “The selected point sits in an elevated safety overview cell citywide.”

It should not let overview dominate the report.

---

14. Frontend Integration Changes Required

14.1 Current obvious mismatch

The current frontend still works on a legacy mock type:

• scores
• aiSummary
• evidence

That is not enough for v3.7.5.

14.2 Required frontend-facing types

Frontend must be updated to understand:

• overview response
• detail preview response
• detail final response
• priority actions
• detail items
• building flags
• data gaps

14.3 Current imported demo bug list

These are the most important current frontend mismatches:

1. it does not call canonical backend analysis endpoints yet
2. its map render calls still target demo-only /api/render/*
3. it generates random local scores on click
4. it has no true report-confirm step yet
5. it still treats the right panel as a score-first card instead of action-first preview/final

14.4 Minimum frontend contract update for first integration

The minimum successful integration path is:

1. keep map resource server as-is
2. keep overview tag buttons:
   • general
   • safety
   • transit
   • amenities
3. on click:
   • enter preview state
4. on priority/radius change:
   • call preview endpoint
5. on confirm:
   • call final report endpoint

---

15. Recommended Backend / Frontend Wiring

15.1 Best architecture for the demo phase

Recommended topology:

Browser
  -> local demo server (Node/Express)
     -> static frontend
     -> MBTiles/vector tile endpoints
     -> thin proxy for preview/final analysis if needed
        -> Python analysis backend

15.2 Why this is better than direct browser-to-Python first

Because it gives us:

• easier same-origin frontend development
• less CORS exposure
• cleaner future Pensar story
• preserved separation between map assets and analysis engine

15.3 Canonical ownership remains with Python

Even if Node provides proxy endpoints, the canonical business logic still belongs to Python.

That means:

• Node = adapter/proxy/static layer
• Python = analysis source of truth

---

16. Pensar / Security Implications

16.1 Why the split matters

The current frontend demo imported a permissive local map server.

If we add analysis directly into that same server without discipline, we risk:

• duplicate logic
• loose request validation
• wider attack surface

16.2 v3.7.5 security-friendly choice

The security-friendly design is:

• canonical analysis logic in Python
• thin compatibility layer only
• strict canonical request validation
• minimal accepted radii
• normalized category allowlist

16.3 Direct benefit for future Pensar testing

This makes future Pensar review easier because:

• there is one real analysis core
• compatibility routes are adapters, not shadow logic
• category IDs and radii are bounded

---

17. Provider Strategy

17.1 Direct mode remains the default test path

For current testing:

• use DirectQueryDataProvider

17.2 Skill mode remains preserved

Keep:

• SkillDataProvider
• mode switching
• future auto mode

But:

• do not block frontend/backend integration on Skill readiness

17.3 Why this is still the right choice

It protects progress.

The frontend can integrate now. The backend can test now. Skill can drop in later.

---

18. Known Implementation Tasks After This Document

18.1 Backend tasks

1. finish category rename cleanup everywhere
2. keep detail preview and detail final on the same deterministic core
3. add alias-safe overview handling for general / overall
4. preserve direct-mode reliability
5. optionally expose thin compatibility endpoints or support proxy integration

18.2 Frontend tasks

1. replace mock click analysis with preview call
2. replace mock Generate usage with report-confirm workflow
3. update right panel to consume:
   • priority_actions
   • why_now
   • detail_items
   • report_markdown
4. keep map resource pipeline intact

18.3 Integration tasks

1. verify local preview latency
2. verify final report generation after confirm
3. verify category alias handling
4. verify radii 200/500/1000
5. verify building section only appears when findings exist

---

19. Acceptance Criteria For v3.7.5

v3.7.5 is considered correctly implemented when:

1. frontend category buttons map cleanly to:
   • general
   • safety
   • transit
   • amenities
2. backend canonical categories are:
   • amenities
   • transit
   • safety
   • building
3. click + radius + priority order can produce a preview response without LLM
4. confirm can produce a final report response
5. report explicitly names the selected radius and local context
6. building findings appear only when they exist
7. overview remains non-reporting and non-ranked
8. the product still works if Skill is absent

---

20. Final Direction

v3.7.5 is the version where we stop treating the frontend and backend as separate thought experiments.

From this point onward:

• frontend map resources stay where they already work
• backend owns canonical analysis logic
• category naming follows the frontend
• preview and final report are formally separated
• building handling follows Option 4A
• report writing aims for concrete conclusions, not generic summaries
• overview remains navigation, not personalized reasoning

This is the correct integration version to build against next.