Skip to content

feat: #998 create opening spatial validation endpoint - #1031

Merged
craigyu merged 37 commits into
mainfrom
feat/998-create-opening-spatial-validation-endpoint
Oct 29, 2025
Merged

feat: #998 create opening spatial validation endpoint#1031
craigyu merged 37 commits into
mainfrom
feat/998-create-opening-spatial-validation-endpoint

Conversation

@craigyu

@craigyu craigyu commented Sep 23, 2025

Copy link
Copy Markdown
Collaborator

Description

Closes #998

The backend can now process geojson, gml and xml(esf) files, it returns the validated data or errors.
For esf file, we are only extracting opening definition spatial data and tenure data, others are ignored for now until we know what to do with them.

All test files are located under:
frontend/src/__test__/fixtures


Thanks for the PR!

Deployments, as required, will be available below:

Please create PRs in draft mode. Mark as ready to enable:

After merge, new images are deployed in:

craigyu added 13 commits August 25, 2025 16:28
Introduced Douglas-Peucker vertex thinning for GeoJSON uploads and enforced validation that all features are within the Province of BC boundary using CRS-specific boundary files. Added new DTOs for extracted geo data, updated constants, and set multipart file size limits in application configuration.
Enhanced backend to support GML file upload with CRS detection, geometry validation (no curves), BC boundary checks, and vertex thinning. Updated OpeningEndpoint and OpeningSpatialFileService to return ExtractedGeoDataDto. Added GeoTools GML dependencies. Frontend SpatialUtils now detects and rejects GML files containing curve geometries. Added test GML fixtures for curve, out-of-BC, and non-thinned cases.
Enhanced the backend to robustly extract and parse multiple GML geometries from ESF/XML and GML files, including improved CRS detection, geometry validation, and GeoJSON conversion. Added XML-based geometry extraction and auto-detection of GML2/GML3 for parsing. Updated frontend utility to fallback to ESF XML extraction logic if no features are found in GML, improving compatibility with various GML/ESF file structures. Minor fixture file formatting fixes.
Enhanced backend ESF spatial file processing to extract tenure information and opening metadata, returning them in the ExtractedGeoDataDto. Updated frontend API types and upload logic to handle new response structure. Added new test fixtures for ESF XML files and improved CRS detection and geometry extraction.
Introduces TenureDto to OpenApi models and updates ExtractedGeoDataDto to include a tenureList. Changes openingGrossArea in GeoMetaDataDto from string to number for type accuracy. Refactors esfXmlToGeoJSON in SpatialUtils to improve MultiPolygon and Polygon extraction logic.
Enhanced JavaDoc comments in OpeningSpatialFileService.java for better clarity on method responsibilities, parameters, and exceptions. Added a new ESF/XML test fixture (esf-not-in-bc.xml) to frontend test resources for validating spatial file processing outside BC boundaries.
Replaced fully qualified class names with direct imports for improved readability and consistency. Updated usages of collection and XML-related types to use imported classes, and simplified code by removing redundant package prefixes. No functional changes were made.
@craigyu craigyu self-assigned this Sep 23, 2025
Copilot AI review requested due to automatic review settings September 23, 2025 16:52
@craigyu craigyu linked an issue Sep 23, 2025 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements spatial file validation for opening creation by adding a new backend endpoint to process GeoJSON, GML, and ESF/XML files with comprehensive validation. The frontend integrates this endpoint into the opening creation workflow.

Key changes:

  • Added new backend spatial file processing service with multi-format support and comprehensive validation
  • Created new API endpoint for spatial file upload with structured response containing geometry and metadata
  • Consolidated favorite opening endpoints from separate service into main opening endpoint

Reviewed Changes

Copilot reviewed 34 out of 38 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
frontend/src/utils/SpatialUtils.ts Enhanced spatial processing with curve detection and improved geometry extraction
backend/src/main/java/ca/bc/gov/restapi/results/postgres/service/OpeningSpatialFileService.java New comprehensive spatial file processing service with validation pipeline
backend/src/main/java/ca/bc/gov/restapi/results/postgres/endpoint/OpeningEndpoint.java Added spatial file upload endpoint and consolidated favorite operations
frontend/src/screens/CreateOpening/index.tsx Integrated spatial file upload mutation into opening creation workflow
frontend/src/services/OpenApi/services/OpeningFavoriteEndpointService.ts Removed - functionality moved to OpeningEndpointService
Comments suppressed due to low confidence (3)

frontend/src/utils/SpatialUtils.ts:1

  • This console.log statement should be removed from production code. Use proper logging or remove it entirely.
import proj4 from "proj4";

frontend/src/utils/SpatialUtils.ts:1

  • This console.warn statement should be removed from production code. Use proper logging or remove it entirely.
import proj4 from "proj4";

frontend/src/utils/SpatialUtils.ts:1

  • The condition checks both geojson and file values, but at this point the geojson may not be populated yet since the file upload is asynchronous. This could cause the validation to fail incorrectly.
import proj4 from "proj4";

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread backend/src/main/java/ca/bc/gov/restapi/results/postgres/dto/TenureDto.java Outdated
Introduces a settings.xml file to configure the OSGeo repository and updates the Dockerfile to use this Maven settings during the build process. This ensures dependencies from OSGeo are available when building the backend image.
Updated JavaDoc comments for ExtractedGeoDataDto and TenureDto to provide clearer descriptions of their purpose and fields. Simplified the error message in OpeningSpatialFileService when processing GML files to avoid exposing internal exception details.
Added secure configuration to DocumentBuilderFactory in multiple methods to prevent XML External Entity (XXE) and DTD-based attacks. This includes disabling external entity processing, disallowing DOCTYPE declarations, and enabling secure processing features.
Introduces reprojection of geometries to EPSG:4326 in OpeningSpatialFileService for supported file types, ensuring consistent output CRS. Adds gt-epsg-hsql dependency to backend for CRS support. Updates frontend SpatialUtils to remove EPSG:42102 registration and improves GML naked polygon extraction logic.
@craigyu
craigyu requested a review from Copilot September 23, 2025 18:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 36 out of 40 changed files in this pull request and generated 5 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread frontend/src/utils/SpatialUtils.ts
Comment thread frontend/src/screens/CreateOpening/index.tsx
Comment thread frontend/src/screens/CreateOpening/index.tsx
Updated the analysis workflow to run Maven with the settings.xml file for backend builds. This ensures custom repository or build configurations are applied during CI analysis.
@craigyu
craigyu enabled auto-merge (squash) October 29, 2025 17:09
@craigyu
craigyu merged commit a177eb8 into main Oct 29, 2025
24 of 26 checks passed
@craigyu
craigyu deleted the feat/998-create-opening-spatial-validation-endpoint branch October 29, 2025 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create Opening Spatial Validation Endpoint

5 participants