Feb3 cleanup - #27
Merged
Merged
Conversation
- Fix double-delete bug in RoundGenerationWorkflow: the empty-results branch deleted the round record explicitly, then the compensation handler caught the ValueError and deleted it again. Removed the redundant delete since compensation handles all failure cases. - Add compensation to UnionPixelSamplingWorkflow: child workflows now undo round assignments on failure via remove_round_assignments activity, preventing orphaned partial data. - Add remove_round_assignments activity: inverse of update_round_assignments, strips a round number from coverage_pixel or coverage records. - Remove deprecated _sampled_count parameter from update_campaign_area_sampled_count_for_union.
…flow The parent workflow now returns area_workflow_map linking campaign area IDs to child workflow IDs. When the stratified cluster wizard completes, it passes this map up through RoundsManager to LocationsPage, which feeds it into CampaignAreasManager as externalSamplingWorkflows. The existing polling and shimmer infrastructure picks up each child workflow and shows "Sampling..." on the corresponding campaign area rows.
Replace executemany/for-loop patterns with single UPDATE ... WHERE id = ANY() queries for round assignments. The old approach executed N individual statements per batch (one per coverage record), causing severe performance issues under concurrent load — 1m42s for 50 records on a 790K row table with GIN index. Three locations fixed: - update_round_assignments in rounds.py (executemany → single UPDATE) - remove_round_assignments in rounds.py (executemany → single UPDATE) - assign_pixels_to_round in cluster_sampling.py (for-loop → single UPDATE)
Stores risk category (high_risk, low_risk, hard_to_reach) on campaign areas during stratified cluster sampling. Displays category after area name in the UI. Also fixes bug where CampaignAreasManager didn't reload areas when external sampling workflows arrived, preventing the "Sampling..." shimmer from appearing on newly created areas.
Replace NOT EXISTS subqueries with ON CONFLICT DO NOTHING in create_coverage_pixels_for_union and create_coverage_pixels_for_campaign_area (62s → <1s for ~1300 rows). Remove unused ST_AsGeoJSON geometry serialization from pixel sampling queries — only lat/lon are needed for adaptive sampling.
Show "Generating..." shimmer on campaign areas during stratified cluster parent workflow, transitioning to "Sampling N areas..." during child workflows. Use silent polling to avoid page flickering. Match category label colors to categorization wizard columns (red/green/yellow).
R function: Accept simple coordinate arrays and uncertainty values instead of GeoJSON FeatureCollection. Remove sf library dependency (st_read, st_centroid, geojson_list). Work directly with coordinate matrix for nn2 nearest-neighbor. Return selected indices instead of full GeoJSON with all features marked. Python side: Build simple arrays instead of GeoJSON dicts. Parse response as selected_indices list. Remove unused ST_AsGeoJSON fetch from location queries (geometry was immediately discarded). Eliminates per-request overhead of: - GeoJSON → JSON string → sf object conversion (st_read) - Per-iteration st_centroid calls on sf objects - sf object → GeoJSON conversion (geojson_list) for all candidates - Brace-counting JSON parser for R stdout extraction - Full GeoJSON response parsing to find selected features
Store workflow status on campaign_areas so the sampling indicator survives page reloads. Use ST_Contains on pixel centroid instead of ST_Intersects on geometry to exclude border pixels. Skip the centroids MVT query in pixels_by_campaign when metadata_field is not needed, halving tile generation work for normal map views.
- Add bounding box filter (&&) back to pixel-area association query alongside ST_Contains centroid check to maintain GIST index usage - Fix R preprocess_params to handle fromJSON returning a matrix for uniform coordinate arrays - Parse OpenFaaS responses with raw_decode to handle stderr mixed into the response body
Use NULLIF to convert empty strings to NULL before casting to UUID, preventing db errors when frontend sends indicator_id as empty string.
Documents the ALTER SYSTEM settings and ANALYZE commands needed to optimize tile query plans for the pixels_by_campaign function.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.